1/2 


•A122  146  BEHRING  LUBRICANT  INTERFACE  MONITORING  USING  COMPOSITE 
SIGNATURE  ANHLVSIS(U)  ADAPT  SERVICE  CORP  READING  MB 
H  E  HUNTER  ET  AL.  SEP  82  ADAPT-82-5  N00014-82-C-0145 

F/G  13/9  NL 


UNCLASSIFIED 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS- 1963-A 


ADAPT  REPORT  NO.  82 


BEARING  LUBRICANT  INTERFACE  MONITORING 

USING  COMPOSITE  SIGNATURE  ANALYSIS 


PREPARED  BY 

HERBERT  E.  HUNTER 

AND 

DAVID  C.  HUNTER 


QWi 

—D  CORPORATION 


ADAPT  SERVICE  COP 
P . 0 .  BOX  5t 
READING  MASS .  C 


SEPTEMBER  1982 

FINAL  REPORT  FOR  PERIOD  1  JAN.  1982  THRU  31  JUL .  1982 


fb 


LU 

i..q 


PREPARED  FOR: 

OFFICE  OF  NAVAL  RESEARCH 
800  N.  QUINCY  STREET 
ARLINGTON,  VIRGINIA  22217 

AND 

NAVAL  AIR  DEVELOPMENT  CENTER 
HARMINSTER,  PA  18974 


s 


DTIC 

ELECTE 
DEC  7  1982 


10 


D 


UNDER  l v  0  6 

CONTRACT  N00014-82-C-0145 


003 


i  miUTiu-.  . 


.  a 

T. 


i- 


M IL  -  STD- 847  A 
31  January  1973 


SECURITY  CLASSIFICATION  OF  THIS  FACE  Pata  Entered) 


REPORT  DOCUMENTATION  PAGE  bkfote°comK>;™S!Vcp.m 

N£PONT  NUM0CR  ll.  COVT  ACC ES5»ON  NO.  L  RCCIf'lCN  CATALOG  nJmuCA 


4.  TITLE  (ami  SmtUlta) 

Bearing  Lubricant  Interface  Moni¬ 
toring  using  Composite  Signature 
Analysis _ _ _ 

1.  author?.* 

Herbert  E.  Hunter 
David  C.  Hunter 

•.  nMtMM  ORO.NI1.TION  NAMC  MO  aoorcs* 

ADAPT  Service  Corporation 
P.  O,  Box  58 


I.  Type  OP  .«PORT  *  PERIOD  COVIRCO 


Final  Report 


1  Jan 


Repor 

1982-3 


lJul  19  8t 


S.  REAF0NMIN6  OAO.  AEAOAT  NOW  SEA 

82-5 _ 

a.  contract  oa  Caant  nuwoca/*) 


N00014-82-C-0145 

TO.  PROGRAM  ELEMENT. VsOJ'iCT.  T  ASK 
AAfi  A  S  VO  AN  UNIT  NUMBERS 


It.  CONTROLLING  OFFICE  NAME  AMO  AOOA6SS  »*•  RE  AO  AT  OAT* 

Office  of  Naval  Research  Sept  1982 _ 

8O9  N.  Quincy  St.  ” 

tUtarant  tram  Cantrolltnt  oifiem)  |  IS.  ICC  J  R1T  fl/S/*  eapati) 


MONlfoAMTO  AGENCY  nSSt*  a{ 


Naval  Air  Development  Center 
Warminster,  Pa.  18974 

l«.  OtSTRiawTIOH  ITATCMCHT  IR  Pit.  Rpn|  . 


Unclassified _ 

TZ  OECLASSIFICATIOk/OOVNOAAOIMO 
SCHEDULE 


distribution  statement  a 

Approved  for  public  relecuM|  > 
Distribution  Unlimited 

it.  MsrmauTiOM  itatcmcmt  <•!  i*.  mmm  .t.*.  m  u«t  jo.  u  <m««i  m.  »iw<i) 


I*.  MPPI.eMSNT.Rr  notes 


•».  ksvporm.  Eigenvectors,  Orthogonal  Functions, 

Maintenance,  Failure  Detection,  Wear  Metal  Contamination, 
Oil  Analysis,  Bearing  Lubricant  Interface,  Incipient 
Failure,  Engine  Health  Monitoring,  Pattern  Recognition, 


ifSTIIACf  eiwNNS  erHf  fVFmifr by  oTm.-S  fiv^ier;  •* 

This  report  presents  the  results  of  the  feasibility 
phase  of  a  defense  small  business  advance  technology  program 
(DE8AT) .  The  ultimate  objective  of  the  program  is  to  improve 
and  simplify  engine  failure  prediction.  This  is  to  be  accom¬ 
plished  by  developing  a  series  of  objective  algorithms  which 
can  use  the  results  of  standard  engine  monitoring  measure¬ 
ments  applicable  to  the  bearing  lubricant  interface  to  detect. 


OO  1  jmTti  1473 


COITION  OF  I  NOV  ft*  IS  O0SOLCTC 


SECURITY  CLAJilMv  a  T  1 0 1 1  OF  This  P«.v;4.  ,«»•**  Data  IwifN) 


MIL-STD-847  A 
31  January  1973 


ttCUWTY  CL<UiyiC«TIB>  Of  THU  AAOtfWH..  On.  tuft**) 

Identify  and  estimate  the  remaining  life  of  worn  engine 
components.  The  Phase  I  objectives  which  were  successfully 
achieved  and  reported  here  were  tot  1)  identify  the  data 
which  is  currently  available,  2)  establish  the  procedures 
required  to  obtain  it,  and  3)  to  develop  demonstration  algorithm^ 
for  a  single  aircraft  engine  type  which  would  define  .he 
expected  improvement  in  anticipating  engine  failures  and 
illustrate  the  use  of  this  approach.  Specifically,  of  the 
12  failures  occurring  in  the  data  set  available  for  the  feasi¬ 
bility  study,  three  were  anticipated  by  current  maintenance 
monitoring  procedures  and  the  remaining  nine  all  were  found 
as  a  result  of  preflight  inspection  or  caused  preflight  or 
inflight  aborts.  The  algorithms  developed  as  part  of  this 
feasibility  study  would  have  anticipated  all  three  of  the 
failures  identified  by  the  present  methods  and  would  also 
have  found  seven  of  the  nine  failures  which  were  missed  by  the 
current  methods.  Thus,  assuming  that  the  sample  of  engines 
utilized  in  this  study  is  typical,  current  methods  are  anticipat¬ 
ing  approximately  a  quarter  of  the  failures.  Implementation  of 
the  failure  detection  algorithms  developed  in  this  study  should 
increase  this  by  a  factor  of  three  to  the  detection  of  approxi¬ 
mately  three  quarters  of  the  engine  failures.  Diagnostic,  and 
time  to  failure  algorithms  are  also  demonstrated,  illustrations 
are  provided  showing  how  the  unique  products  of  an  eigenvector 
analysis  can  provide  more  sophisticated  analysis  tools  which 
may  improve  the  Phase  I  performance  even  farther.  The  use  of 
these  algorithms  in  a  "user-friendly"  automated  system  is 
described. 
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1.0  INTRODUCTION 

V 

This  report  presents  the  results  of  the  feasibility 
phase  or  Phase  I  of  a  defense  small  business  advance  technology 
program  (DESAT) .  The  ultimate  objective  of  the  program  is  to 
improve  and  simplify  engine  failure  prediction.  This  is  to 
be  accomplished  by  developing  a  series  of  objective  algorithms 
which  can  use  the  results  of  standard  engine  monitoring  measure- 
ments  applicable  to  the  bearing  lubricant  interface  to  detect, 
identify  and  estimate  the  remaining  life  of  worn  engine  components. 
The  Phase  I  objectives  which  were  performed  in  the  study  which 
is  reported  here  were  to:  1)  identify  the  data  which  is 
currently  available,  2)  establish  the  procedures  required  to 
obtain  it  and  3)  to  develop  demonstration  algorithms  for  a 
single  aircraft  engine  type  which  would  define  the  expected 
improvement  in  anticipating  engine  failures  and  illustrate 
the  use  of  this  approach. 

At  present,  there  are  a  number  of  different  programs  under 
which  measurements  are  performed  which  would  be  useful  in  moni¬ 
toring  the  bearing  lubricant  interface  in  particular  and  the 
engine  health  in  general.  The  five  sets  of  these  measurements 
which  were  considered  in  this  study  were:  1)  the  engine  test 
cell  measurements  performed  after  major  overall  of  the  engine, 

2)  the  concentrations  of  metallic  impurities  in  the  oil  measured 
as  part  of  the  Naval  Oil  Analysis  Program  (NOAP) ,  3)  the  Navy 
Maintenance  and  Material  Management  information  reports 
(3M),  4)  the  Inflight  Engine  Condition  Monitoring  System  (IECMS) 
data,  and  5)  preflight  inspection  data. 

Upon  the  successful  completion  of  Phase  I  which  is  reported 
in  this  report,  it  is  anticipated  that  a  Phase  2  program  will 
be  implemented  with  the  objectives  of:  1)  developing  and 
executing  a  demonstration  that  will  prove  the  gain  suggested 
by  these  feasibility  studies,  2)  refining  the  accuracy  of  the 
estimated  performance  gains  and  3)  providing  performance  gain 
versus  cost  trade-off  information.  The  Phase  2  plans  will 
be  the  subject  of  a  separate  document,  however,  the  present 
report  provides  the  basis  upon  which  these  Phase  2  plans  will 
be  developed. 

The  present  report  begins  with  a  summary  of  the  major 
results,  conclusions  and  recommendations.  This  summary  presented 
in  Section  2.0  shows  the  results  of  the  feasibility  study  without 
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presenting  a  description  of  the  methodology  or  justifying 
the  results.  The  recommended  incipient  failure  detection  and 
analysis  system  resulting  from  this  study  is  also  included  at 
the  end  of  this  summary  of  results,  conclusions  and  recommenda¬ 
tions.  The  description  methodology  and  justification  of  the 
results  is  presented  in  the  remaining  sections  and  the  appendices. 
Section  3.0  describes  the  data  acquisition  and  preparation 

which  was  required  for  the  feasibility  study.  This  is  followed 
by  a  section  describing  the  development,  analysis  and  expected 
performance  of  the  failure  analysis  algorithms.  The  details 
of  the  ADAPT  approach  to  empirical  analysis  are  given  in  the 
appendices. 


j 
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2.0  SUMMARY  OF  RESULTS,  CONCLUSIONS  AND  RECOMMENDATIONS 

The  major  result  of  this  study  is  the  verification  of 
our  anticipated  result  stated  in  the  Phase  I  proposal  as, 

"At  the  conclusion  of  the  Phase  1,  we  will  have  shown  the 
feasibility  of  improving  bearing  lubricant  interface  wear  moni¬ 
toring  and,  thereby,  reducing  the  number  of  castrophic  engine 
failures  by  signature  analysis  of  composite  signatures  of 
several  wear  sensor  measurements".  Specifically,  of  the  12 
failures  occurring  in  the  data  set  available  for  the  feasibility 
study,  three  were  anticipated  by  current  maintenance  monitoring 
procedures  and  the  remaining  nine  all  were  found  as  a  result 
of  preflight  inspection  or  caused  preflight  or  inflight  aborts. 

The  algorithms  developed  as  part  of  this  Phase  1  feasibility 
study  would  have  anticipated  all  three  of  the  failures  identified 
by  the  present  methods  and  would  also  have  found  seven  of  the 
nine  failures  which  were  missed  by  the  current  methods.  Thus, 
assuming  that  the  sample  of  engines  utilized  in  this  study  is 
typical,  we  conclude  that  current  methods  are  anticipating 
approximately  a  quarter  of  the  failures  and  that  implementation 
of  the  failure  detection  algorithms  developed  in  this  study  should 
increase  this  by  a  factor  of  three  to  the  detection  of  approxi¬ 
mately  three  quarters  of  the  engine  failures.  Clearly,  this  is 
a  significant  improvement. 

In  addition  to  the  improvement  in  the  performance,  the 
proposed  algorithms  will  be  implemented  in  a  fully  automated 
and  interactive  mode.  Information  is  currently  available  from 
the  individual  measurement  programs  usually  in  the  form  of  raw 
data.  The  proposed  algorithms  will  automatically  combine  all 
of  this  information  and  output  an  answer  instead  of  the  results 
of  the  testing.  For  example,  in  the  oil  program,  the  processing 
and  recording  of  the  concentrations  of  the  oil  sample  is  handled 
entirely  by  automated  test  equipment  and  computers.  The  result 
is  a  printed  table  of  the  concentrations  of  the  metallic 
impurities  plus  a  “lab  advice"  code.  If  the  ADAPT 
algorithms  were  utilized,  this  same  information 
would  be  combined  with  the  results  of  the  test  cell 
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run  at  the  time  the  engine  was  last  overhauled  (also  processed 
and  recorded  by  computers  for  many  engines*)  and  applicable 
3M  data  in  the  computer  .  The  result  would  be  a  print-out 
indicating  that  the  engine  was  healthy  or  that  a  failure  could 
be  anticipated  and  for  the  more  common  failures  a  diagnosis 
of  the  most  likely  component  to  fail.  In  many  cases,  an  estimate 
of  the  time  remaining  before  the  failure  occurs  will  also  be 
made. 

This  answer  will  be  followed  by  the  display  of  a  menu 
which  the  user  may  use  to  obtain  additional  information  if 
desired.  Thus,  from  the  users  point  of  view,  the  proposed 
approach  will  yield  answers  which  require  no  further  analysis 
to  determine  the  corrective  action,  if  any,  which  is  required. 
Along  with  this  gain  in  user  "friendliness",  one  can  anticipate 
finding  three  tiroes  as  many  failures  as  are  currently  being 
found. 

2.1  Conclusions 

Considerable  additional  detail  of  these  results  as  well 
as  the  development  of  many  other  significant  conclusions  are 
described  in  Section  4.0,  Development  Analysis  and 
Expected  Performance  of  Failure  Analysis  Algorithms.  Specifi¬ 
cally,  the  development  and  analysis  of  the  expected  performance 
has  shown: 

1)  The  data  currently  obtained  by  existing  measurement 
programs  contains  much  unused  information  which  is  sufficient 
to  jtriple  the  number  of  failures  anticipated  relative 

to  current  procedures.  It  is  anticipated  that  this  will  result 
in  a  very  significant  reduction  in  preflight  and  inflight  aborts 
as  well  as  reduce  the  number  of  problem  engines  which  leave 
the  test  cell. 

2)  The  data  required  to  develop  these  algorithms  and  to 
apply  them'is  available  from  standard  Navy  data  archival  sources. 

3)  Most  engine  failures  called  by  the  ADAPT  algorithms  are 
called  20  to  160  engine  hours  in  advance  of  the  actual  occurrence 
of  the  failure. 

4)  A  demonstration  algorithm  for  predicting  the  time  to 
failure  F3  was  developed  and  showed  a  capability  to  predict  the 
time  at  which  the  failure  would  occur  from  160  engine  hours 

in  advance  of  the  failure  to  the  occurrence  of  the  failure  with 
a  3  sigma  error  of  ±  18  engine  hours. 

*  At  Norfolk,  o»">e  automated  test  cell  became  operational  in  Jan. 

1982  and  the  second  is  expected  in  the  near  future. 


5)  Both  simple  to  use  go-no-go  algorithms  which  printout 
the  engine  health  and  the  action  required  and  more  sophisticated 
lab  analysis  procedures  were  demonstrated. 

6)  The  algorithms  required  for  both  the  simple  go-no-go 
procedures  and  the  more  sophisticated  analysis  procedures 

can  all  be  implemented  in  any  one  of  three  ways  from  a  hardware 
standpoint.  These  three  ways  are:  1)  on  the  existing  NOAP 
computer,  2)  on  the  existing  computer  in  the  automated  test 
cell  or  3)  on  a  stand  alone  low  cost  microcomputer. 

7)  As  a  result  of  the  analysis  of  the  relative  importance 
vectors,  we  have  concluded  that  there  is  a  cleanup  mechanism 
which  we  were  previously  unaware  of  which  allows  the  metallic 
contaminants  to  indicate  the  presence  of  non-metallic  contaminants. 

8)  It  can  be  anticipated  that  the  addition  of  a  larger 
training  set  of  engines  will  improve  the  overall  performance 

of  these  algorithms  relative  to  that  shown  in  these  feasibility 
studies. 

9)  A  significant,  but  expected  difference  was  observed 
between  data  vectors  constructed  from  manual  and  automated 
test  data.  This  difference  was  sufficiently  small  that  it 
does  not  mask  thet  engine  health  information. 

2.2  Recommenda t ions 

Based  on  successful  achievement  of  all  of  the  Phase  I 
objectives  and  demonstration  of  the  potential  for  signficant 
improvement  in  detecting  potential  aircraft  engine  failures, 
it  is  recommended  that  Phase  2  of  the  bearing  lubricant  inter¬ 
face  health  monitoring  using  integrated  composite  signature 
analysis,  be  implemented  and  directed  towards  achieving  the 
following: 

1)  Develop  and  implement  a  six  month  demonstration  of 
the  recommended  incipient  failure  detection  system  for  the 
TF30  engine  at  Norfolk  Naval  Air  Station.  Specifically,  this 
would  include:  a)  incorporation  of  revised  incipient  failure 
detection  and  diagnostic  algorithms  in  either  the  NOAP  computer, 
the  test  cell  computer  and/or  a  stand  alone  microcomputer, 
b)  the  integration  of  these  procedures  into  the  existing  organi¬ 
zation,  c)  the  presence  of  ADAPT  personnel  during  the  six  month 
demonstration  to  insure  proper  implementation  and  train  Navy 
personnel  in  its  use  and  d)  the  evaluation  of  the  effect  of  the 
implementation  of  both  the  fully  automated  system  and  the  de¬ 
tection  and  analysis  support  system  on  the  number  of  the  inflight 
and  preflight  aborts  occurring. 
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2)  To  build  a  cost  effectiveness  computer  model  to  demonstrate 
the  usefulness  of  this  concept  for  other  engines,  data  sets  and 
facilities. 


3)  To  integrate  the  Inflight  Engine  Condition  Monitoring 
System  (IECMS)  data  into  the  analysis  to  determine  what  improve¬ 
ment  is  possible  for  the  IECMS  equipped  aircraft  using  the  data 
used  for  this  study  and  the  ADAPT  data  analysis  procedures. 

4)  Perform  analysis  of  relative  importance  vectors  to 
determine  how  the  costs  may  be  reduced  and  performance  improved 
by  modifying  the  data  collection  and  recording  procedure. 

2.3  Recommended  Incipient  Failure  Detection  and  Analysis 
Demonstration  System 

The  incipient  failure  detection  demonstration  software 
packages  will  be  implemented  in  the  existing  NOAP  data 
gathering  hardware  and/or  the  automated  test  cell.  They 
will  contain  two  completely  independent  user  oriented  systems: 

1)  the  automated  failure  detection  system  and  2)  the  detection 
and  analysis  support  system.  They  could  also  be  implemented 
in  a  low  cost  stand  alone  microcomputer,  but  this  is  not 
recommended  unless  an  unforeseen  problem  arises  with  implementing 
it  in  the  existing  hardware.  To  illustrate  the  concept,  we 
shall  describe  a  system  implemented  in  the  NOAP  data  gathering 
hardware.  The  modification  required  for  implementation  in  the 
automated  test  cell  or  a  stand  alone  unit  would  be  straight¬ 
forward  and  would  not  change  this  description  significantly 
from  the  users  viewpoint. 

The  system  would  incorporate:  1)  a  capability  to  use 
the  existing  NOAP  data  files,  2)  input  the  test  cell  results 
(preferably  directly  in  computer  compatable  format) ,  3)  accept 
computer  comp  .able  copies  of  the  3M  data  being  submitted  by 
NAS  Norfolk  to  the  3M  data  base,  4)  the  ADAPT  derived  algorithms 
and  5)  the  ADAPT  interactive  scatter  plot  and  relative  importance 
vector  analysis  tools.  As  each  new  test  cell  data  set  or  oil 
sample  is  added,  this  software  package  would:  1)  update  the 
data  vector  for  the  engine  effected,  2)  apply  the  algorithms 
to  the  updated  data  vectors,  3)  printout  a  message  defining 
the  engine  health  and  4)  display  a  menu  which  the  user  may  use 
to  request  additional  information  and  analysis. 

The  message  defining  the  engine  health  would  state  either 
failure  is  anticipated  or  a  failure  is  not  anticipated.  If 
a  failure  is  anticipated,  the  printout  would  also  include  the 
results  of  the  application  of  the  diagnostic  and  time  to  failure 
algorithm.  For  most  failures,  this  would  define  the  system 
where  the  failure  is  most  likely  to  occur.  For  the  more 
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ion  failures,  the  exact  failure  should  be  defined  and  in 
cases  the  expected  engine  hours  left  to  failure  would  be 
ted  out. 

Thus,  instead  of  the  current  printout  of  test  results 
i  a  suggested  lab  advice  for  the  NOAP  data  and  separate 
tout  of  the  test  cell  results  at  the  test  cell,  the  user 
d  see  a  printout  such  as:  "Engine  Serial  No  -701251  on 
Serial  No-160899  shows  no  indication  of  incipient  failure 
d  on  test  cell  data  of  15  July  1983  and  oil  samples  taken 
ugh  21  September  1983". 

If  an  engine  failure  were  about  to  occur,  the  user  would 
lerted  by  a  message  of  the  foilwing  form: 

***CAUTION***  Engine  Serial  No-701251  is  expected  to 
in  the  near  future.  Analysis  indicates: 

1)  Failure  will  occur  in  the  afterburning  fuel  system. 

2)  Most  likely  component  to  fail  is  the  injector  nozzle 
a  probability  of  0.76. 

3)  If  the  failing  component  is  the  injector  nozzle,  the 
ined  probability  that  the  engine  will  operate  an  additional 
ours  without  failure  is  0.75'. 

Either  of  the  preceding  messages  will  be  followed  by  a 
pt  asking  if  the  user  desires  additional  information.  If 
user  replies  In  the  affirmative,  a  menu  will  be  displayed 
which  the  user  may  elect  to  display  and/or  print  any  or 
of  the  pertinent  test  cell,  NOAP  or  3M  data  as  well  as  the 
T  scatter  plot  or  relative  importance  vector  analysis 
lays  which  are  described  in  Section  4.1  of  this  report. 

selection  will  be  accomplished  by  placing  the  cursor  on 
desired  menu  item  and  executing  a  carriage  return.  Thus, 
ng  skill  will  not  be  important  to  using  this  system. 
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3.0  DATA  ACQUISITION  AND  PREPARATION 


This  section  summarizes:  1)  the  selection  of  the  data, 

2)  the  verification  performed  on  the  data,  and  3)  the  integra¬ 
tion  of  the  data  from  the  various  sources  into  the  data  vector. 

3.1  SELECTION  OF  DATA 

The  Data  acquisition  study  considered  five  sources  of 
data  which  were  anticipated  to  be  available  from  various 
archiving  systems  within  the  Navy.  These  five  sources  of 
data  were:  1)  test  cell  results  after  engine  overhaul,  2) 

Naval  Oil  Analysis  Programs  (NOAP),  3)  Navy  Maintenance 
Material  Management  (3M)  data  base,  4)  flight  line  and  in¬ 
flight  recorded  data  and  5)  Inflight  Engine  Condition  Monitor¬ 
ing  System  (IECMS)  data.  Review  of  the  availability  and 
applicability  of  these  data  sets  for  the  present  studies 
indicated  that  the  first  three  data  sources  were  readily  available 
and  typical  of  data  that  could  be  found  on  other  engines  and 
at  other  facilities. 

A  review  of  the  procedures  used  at  the  flight  line  to 
record  information  indicated  that  at  the  Norfolk  facility  a 
limited  amount  of  information  was  recorded  concerning  the  number 
of  flights,  exhaust  gas  temperature  and  oil  usage.  However, 
the  procedures  for  these  would  clearly  vary  from  squadron  to 
squadron  and  facility  to  facility  within  the  Navy.  Also,  the 
quality  of  the  data  would  be  dependent  upon  the  individual  who 
was  recording  the  data.  For  these  reasons,  it  was  felt  that 
this  flight  line  information  even  where  it  was  available  at 
Norfolk  would  not  be  a  good  type  of  information  to  be  included 
in  the  study. 


The  IECMS  data  was  not  available  on  any  of  the  aircraft 
at  Norfolk.  The  IECMS  data  is  also  only  available  on  a  limited 
number  of  aircraft  and  the  cost  of  obtaining  IECMS  data  was 
considerably  more  than  the  cost  of  obtaining  the  other  data 
which  was  considered  in  this  study.  Furthermore,  it  appeared 
as  though  the  procedures  required  to  obtain  the  IECMS  data 
required  for  this  study  might  not  be  accomplished  within  the 
six  month  available  for  the  study.  However,  it  must  be  empha¬ 
sized  that  this  IECMS  data  is  archived  and  is  available  for 
future  use,  however,  some  processing  may  be  required  to  obtain 
the  data  in  a  useful  format.  Because  of  the  significantly 
greater  cost  per  aircraft  to  obtain  IECMS  data  under  operational 
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conditions,  the  time  and  resource  constraints  on  the  Phase  1 
study,  it  was  felt  that  the  most  useful  Phase  1  program  would 
he  to  determine  the  improvement  in  maintenance  performance 
which  can  he  achieved  without  the  installation  of  the  IECMS 
system  on  aircraft.  Phase  2  studies  can  he  performed  to 
evaluate  the  additional  improvement  possible  both  relative 
to  the  non- IECMS  equipped  aircraft  using  the  methods  developed 
in  this  study  and  relative  to  the  IECMS  aircraft  which  did 
not  use  the  methods  developed  in  this  study  or  the  additional 
data. 


Thus,  the  Phase  1  studies  constructed  the  data  vector 
using:  1)  the  test  cell  data  obtained  after  the  most  recent 
engine  overall,  2)  the  NOAP  data  and  3)  the  3M  data.  The 
specific  components  to  he  derived  from  each  of  these  three 
data  sources  are  shown  in  Table  1.  The  following  sub¬ 
sections  will  discuss  how  each  of  these  data  were  verified 
and  prepared  for  incorporation  into  the  data  vector. 

3.2  TEST  CELL  DATA 

The  availability  and  usefulness  of  the  test  cell  data 
for  this  study  became  apparent  during  the  ADAPT  Service 
Corporation's  fact  finding  visit  to  the  Norfolk  Naval  Air 
Station  and  overhaul  facilities.  At  the  time  of  this  visit, 
an  automated  data  recording  system  with  a  limited  analysis 
capability  had  just  become  operational  in  one  of  the  two  test 
cells  used  to  perform  tests  on  the  engines  after  the  overhauls. 
It  was  anticipated  that  a  similar  automated  system  would 
become  available  in  the  second  test  cell  within  a  year.  We 
were  also  informed  that  when  an  engine  was  received  for 
overhaul,  the  results  of  the  previous  test  cell  tests  were 
still  attached  to  the  engine's  log.  Since  this  data  would 
be  replaced  with  the  new  test  cell  results,  it  was  the 
standard  procedure  to  remove  the  old  test  cell  data  from  the  log 
destroy  it  and  replace  it  with  the  new  test  cell  information. 
This  procedure  offered  an  excellent  opportunity  to  obtain  the 
test  cell  data  which  would  be  pertinent  to  the  entire  time 
period  between  overhauls  and  for  which  the  NOAP  and  3M  data 
would  also  be  available. 

Thus,  our  recommended  plan  was  to  ask  the  Norfolk  test 
cell  personnel  to  retain  the  old  data  rather  than  destroy  it 
and  send  it  to  us  for  use  in  this  program.  We  would  then 
supplement  this  data  with  the  NOAP  and  3M  data  to  construct 
the  data  vectors.  An  exploratory  set  of  data  was  obtained  to 
test  out  this  system  and  this  information  was  obtained  on  the 
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first  three  engines.  It  was  then  used  to  specify  the 
remainder  of  the  data.  The  system  worked  well  and  it  was 
determined  that  this  method  would  be  used. 

Unfortunately,  when  similar  information  was  requested 
on  a  second  and  larger  set  of  engines,  it  was  discovered 
that  the  old  data  was  no  longer  available  with  the  engines 
arriving  at  the  test  cell.  Thus,  our  procedure  had  to  be 
changed  to  utilize  more  recent  test  cell  data  (i.e.  copies 
of  the  current  test  cell  data)  then  originally  anticipated. 

The  effect  of  this  change  in  the  availability  of  data  wars 
that  only  two  of  the  engines  which  were  used  for  this  study 
had  data  available  over  the  entire  time  period  between  two 
successive  entries  to  the  test  cell.  The  remaining  engines 
had  significantly  shorter  time  periods  of  data  and  often 
all  of  the  data  for  a  given  engine  represented  only  good  or 
only  failing  cases. 

The  automated  test  cell  data  was  archived  from  the  time 
that  the  test  cell  first  became  operational  in  January  1982. 
This  data  was  also  used  in  this  study.  Thus,  the  test  cell 
parameters  were  obtained  from  both  the  automated  system  and 
manual  recorded  data.  A  study  was  performed  to  determine  the 
effect  of  mixing  both  manually  recorded  and  automated  test 
cell  data.  The  conclusions  of  this  study  were  that  although 
there  were  measurable  differences  between  these  data,  they 
were  not  a  detriment  to  the  present  study.  In  fact,  a  large 
portion  of  this  difference  would  probably  disappear  with  a 
larger  sample  of  engines.  These  results  will  be  developed 
and  discussed  in  more  detail  in  Section  4.5  after  we  have 
introduced  the  ADAPT  analysis  techniques  used  for  the  study. 

For  both  the  automated  test  cell  and  the  manually  recorded 
data,  the  information  was  supplied  in  hardcopy  format.  Thus, 
all  of  the  variables  used  were  hand  punched.  For  a  number  of 
the  manually  recorded  cases,  there  was  also  the  problem  of 
missing  data.  To  minimize  the  effect  of  keypunching  errors 
and  to  account  for  missing  data,  all  of  the  data  after 
punching  was  processed  through  a  program  which  tested  the 
data  for  missing  values  and  for  unusual  values  indicating 
keypunching  errors.  The  program  Incorporated  in  it  the  nominal 
performance  curves  for  the  engine.  These  nominal  performance 
curves  were  used  to  construct  the  difference  parameters  between 
the  expected  and  observed  performance  which  are  included  in 
variables  No.  80  through  103  in  Table  1.  For  those  cases 
where  data  was  missing,  these  curves  were  also  used  to  estimate 
the  missing  data.  In  this  case,  the  associated  difference 
variable  would  be  zero.  However,  by  using  this  approach. 
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missing  data  did  not  create  large  perturbations  in  the 

data  vector  and  thus  the  effect  of  missing  data  was  minimized. 

To  further  minimize  the  effect  of  potential  keypunch 
errors,  all  of  the  data  was  plotted  on  performance  curves 
such  as  those  shown  in  Figures  1  through  6.  On  these  curves, 
each  of  the  symbols  represents  an  engine  at  a  specified  test 
condition.  The  variables  used  in  the  data  vector  were  those 
obtained  at  the  idle,  intermediate.  Zone  3  and  maximum  after* 
burner  test  conditions.  Thus,  for  each  of  the  engines,  there 
are  four  points  on  the  performance  curves.  The  cross  hatched 
area  on  Figures  1  through  6  shows  the  acceptable  performance 
as  defined  by  the  manufacturer  for  the  TF30  engine.  The 
solid  line  is  the  nominal  performance  which  was  used  to  cal¬ 
culate  the  differences  appearing  in  variables  80  through  103 
on  Table  1. 

By  examining  preliminary  plots  such  as  those  shown  in 
Figures  1  through  6,  any  significant  keypunched  errors  could 
be  detected  and  corrected  prior  to  construction  of  the  data 
vectors.  This  was  accomplished  and  the  results  presented  in 
Figures  1  through  6  are  the  performances  after  the  significant 
keypunch  and  data  omissions  have  been  corrected.  It  should 
be  noted  that  the  acceptable  performances  shown  on  these 
figures  by  the  cross  hatched  bands  are  those  without  the  after 
burner  operating.  For  some  of  the  parameters,  especially  the 
fuel  consumption  shown  in  Figure  6,  this  performance  is  very 
different  for  the  half  of  the  cases  shown  during  which  the 
after  burner  was  operating.  In  fact,  in  this  figure  the  three 
clusters  occurring  on  the  right  hand  side  are  those  associated 
with  no  after  burner  operating  those  with  the  after  burner  in 
Zone  3  and  the  highest  £uel  rate  those  with  the  maximum  after 
burner. 

3.3  NOAP  DATA 

The  NOAP  data  was  converted  from  it's  archival  form  to 
a  format  which  is  compatable  with  the  ADAPT  programs.  This 
involved  a  conversion  of  the  data  to  the  IBM  internal  machine 
format  and  the  arranging  of  the  data  by  engine,  sorted  on  the 
date  and  engine  hours  since  overhaul.  Throughout  the  data 
acquisition  process,  the  engine  hours  since  overhaul  is  used 
as  the  reference  time  frame  and  often  referred  to  as  simply 
engine  hours. 

After  the  NOAP  data  has  been  changed  to  the  ADAPT  format, 
the  second  phase  of  the  data  acquisition  is  the  self  check 
phase.  The  verification  of  the  NOAP  data  is  done  in  three 
ways:  1)  the  clear  and  orderly  displaying  of  the  data,  2) 
software  scanning  for  logical  inconsistencies  and  3)  intelli- 


FIGURE  -  1 


PERIOQHAHCE  CURVE  1T5/1MT2  VS.  PT7/PTC 


FIGURE  -  3 

PERFORMANCE  CURVE  PSJ/PT2  VS.  PT7/PT2 


FIGURE  -  5 

PERFORMANCE  CUBV-  N2/SQ1 THT2 1  VS.  PT7/PT2 
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gent  automated  correction  of  data. 

The  NOAP  data  is  displayed  in  a  concise  chronological 
report  so  that  any  abnormalities  can  easily  be  spotted.  To 
aid  in  spotting  abnormalities,  each  oil  sample  is  tested  for 
specific  logical  faults,  they  are:  1)  engine  transfer  to  a 
second  aircraft  while  in  operation,  2)  unusual  aircraft  type, 

3)  unusual  engine  type,  4)  negative  engine  hour  progress  and 
5)  the  change  in  engine  hours  since  last  overhaul  greater  than 
ten  hours  different  from  the  change  in  engine  hours  since  last 
oil  change.  The  flagging  of  one  or  more  of  these  conditions 
does  not  necessarily  indicate  a  bad  sample;  for  example,  if 
the  hours  since  last  oil  change  decreases  while  the  hours  since 
last  overhaul  increases  an  oil  change  probably  occurred. 

This  information  is  used  to  calculate  a  date  for  the  most  recent 
oil  change  throughout  the  engines  life.  It  was  found  that  the 
NOAP  data  often  had  "unknown1*  values  for  the  hours  since  overhaul 
and  oil  change  as  well  as  the  days  in  transit.  The  program  scans 
for  these  "unknown"  values  and  replaces  them  with  either  the 
average  value  or  interpolates  the  value  between  neighboring 
values.  The  program  will  also  determine  if  the  hours  since 
overhaul  and  hours  since  oil  change  have  been  interchanged  and 
correct  for  that  error.  Whenever  one  of  these  corrections  is 
made,  the  corrected  item  is  flagged  to  indicate  the  correction. 

Figure  7  is  an  example  of  the  NOAP  data  report  generated 
in  the  data  acquisition  phase  for  each  engine  there  are  two 
sections:  1)  the  sample  summary  section,  in  this  section  the 
environment  of  each  sample  is  displayed  as  well  as  the  warnings 
and  flags  discussed  above  and  2)  the  wear  metal  concentration 
section.  In  this  section,  the  date,  engine  hours  and  wear 
metal  concentrations  (ordered  left  to  right  the  concentration  are 
Fe,  ng,  nl,  Cr,  Cu,  Mg,  Ti,  Pb,  Sn,  Ni,  Mo,  Si,  Na,  Ba,  Cd,  Mn,  V, 
zxi,  are  displayed  chronologically. 

In  Figure  7,  for  sample  Number  422,  the  lab  code  is  blank 
in  the  sample  summary  section.  This  indicates  the  data  was 
obtained  from  a  NOAP  computer  summary  sheet  produced  at  Norfolk 
and  the  last  six  mineral  concentrations  were,  therefore,  not 
available.  The  six  values  shown  in  the  wear  metal  concentration 
section  for  sample  422  were,  therefore,  obtained  from  the 
average  of  test  cell  concentrations.  In  future  studies,  the 
numbers  could  be  refined  by  using  a  larger  data  set  and 
averages  appropriate  to  the  case.  However,  it  is  anticipated 
that  this  problem  will  not  exist  in  phase  II  since  the 
data  archived  in  San  Antonio  has  all  18  measurements.  The 
Norfolk  data  was  used  in  the  present  study  because  of  the  time 
lag  in  the  San  Antonio  data  combined  with  the  tight  schedule 
of  this  program  and  the  limits  on  the  availability  of  test 
cell  data. 
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FIGURE  -  7 

Sample  Summary  Section 


ENGINE  SERIAL  NUMBER:  701264  AIRCRAFT  SERIAL  NUMBER: 
415  ANALYSIS  DATE:  2064  3/25/82 

SAMPLE  OATE:  2083  3/24/6.. 

UB  CODE:  ANE 

HRS.  SINCE  OVERHAUL:  605 
HRS.  SINCE  OIL  CHANGE:  0 

OATE  OF  LAST  OVERHAUL:  82083.0 
OATE  OF  LAST  OIL  CHANGE:  82083.0 
REASON:  TEST  CELL 
LAB  RECOMEMDATION:  A 
OATS  IN  TRANSIT:  1 

#MHARHING^**R  AIRCRAFT  TYPE  IS  TEST  NOT  F14A 


135010 


418  ANALYSIS  OATE:  2141  5/21/62 

SAMPLE  OATE:  2138  5/18/82 

LAB  COOE:  ANE 

HRS.  SINCE  OVERHAUL:  666 
HRS.  SINCE  OIL  CHANGE:  62 
OATE  OF  LAST  OVERHAUL:  82083.0 
OATE  OF  LAST  OIL  CHANGE:  82083.0 


REASON:  ROUTINE 
LAB  BECOME NOATION:  A 
OATS  IN  TRANSIT:  3 

-  irsw*  -sssmsr 0,w'’ 


420  ANALYSIS  DATE:  2159  6/  6/82 

SAMPLE  OATE:  2158  6/  7/82 

LAB  COOE:  ANE 

HRS.  SINCE  OVERHAUL:  676 
HRS.  SINCE  OIL  CHANGE:  72 

OATE  OF  LAST  OVERHAUL:  82083.0 
OATE  OF  LAST  OIL  CHANGE:  82083.0 
REASON:  ROUTINE 
LAB  RECOMENDATION:  A 
DAYS  IN  TRANSIT:  1 
SAMPLE  tWMBER:  86 

422  ANALYSIS  iiATE:  2180  6/29/82 

SAMPLE  OATE:  2176  6/25/82 

LAB  COOE: 

HRS.  SINCE  OVERHAUL:  683 
HRS.  SINCE  OIL  CHANGE:  79 

OATE  OF  LAST  OVERHAUL:  82083.0 
OATE  OF  LAST  OIL  CHANGE:  82083.0 
REASON:  ROUTINE 
LAB  RECCMENOATION:  A 
OATS  IN  TRANSIT:  4 
SAMPLE  NUMBER:  0103 


Wear  Metal  Concentration  Section 


415  82003.0 
418  62138.0 
420  82158.0 
422  62176.0 


0.0 

605.0 

62.0 

666.0 

72.0 

676.0 

79.0 

683.0 

1.0 

0.0 

0.0 

1.0 

0.0 

0.0 

3.0 

1.0 

0.0 

1.0 

0.0 

0.0 

0.0 

0.0 

1.0 

0.0 

0.0 

1.0 

0.0 

0.0 

5.0 

0.0 

0.0 

1.0 

1.0 

0.0 

8.0 

1.0 

1.0 

7.0 

1.0 

0.0 

6.0 

1.0 

0.0 

7.0 

1.0  1.0 

0.0  1.0 

0.0  0.0 

0.0  0.0 


1.0  0.0 
2.0  0.0 
3.0  2.0 
3.0  i.l 


0.0 
0.0 
0.0  2 
0.1 
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W  N  Ql  4 


Samples  412  and  418  in  the  sample  summary  section  both 
ve  warning  examples,  sample  415  indicates  that  the  air- 
aft  type  indicated  was  ,l  +  sst"  instead  of  the  normal  F14A  . 
ds  warning  generally  occurs  on  test  cell  samples  and  this 
se  is  not  an  exception  since  the  reason  for  the  sample  is 
splayed  as  "TEST  CELL".  The  warning  occurring  on  sample 
8  occurred  because  when  the  test  cell  was  performed  the 
rcraft  the  engine  was  to  be  placed  in  was  not  known, 
erefore,  the  correct  aircraft  serial  number  could  not  be 
ed. 


The  automated  data  acquisition  program  as  described 
ove  was  used  in  conjunction  with  a  manual  two  pass  technique 
ere  the  results  from-  the  automated  first  pass  was  studied 
d  then  corrected  where  necessary  using  information  from 
e  test  cell  and  3M  data. 

4  3M  DATA 

The  3M  data  was  converted  to  the  IBM  internal  machine 
rmat  and  sorted  on  engine  and  date  in  the  process  of  data 
quisition.  A  concise  summary  of  the  3M  data  was  printed 
r  each  maintenance  entry,  each  entry  printed  the  work  unit 
ading,  the  work  unit  subheading  and  the  work  unit  title  as 
n  be  seen  in  the  example  Figure  8.  There  are  multiple 
tries  for  the  "HOW  MALFUNCTIONED  CODE",  the  "ACTION  TAKEN 
DE"  and  the  "WHEN  DISCOVERED  CODE**  These  occur  because  the 
des  appear  at  several  locations  in  the  archival  records. 

The  3M  data  is  scanned  by  the  software  for  the  following 
gical  faults:  1)  engine  transfer  to  a  second  aircraft 
lie  in  operation  and  2)  work  unit  information  missing.  If 
e  of  these  faults  is  discovered  a  warning  message  is  printed 
thin  the  maintenance  entry  it  was  found. 

5  INTEGRATION  INTO  TRUTH  DATA  AND  DATA  VECTOR 

The  final  step  in  the  data  acquisition  process  is  to 
mbine  the  three  sources  of  data  into  the  data  vector  shown 
Table  1.  This  data  vector  is  created  for  any  date  requested 
om  the  available  data  on  the  engine  requested.  Three  steps 
e  used  in  the  creation  of  the  data  vector.  First  the  different 
urces  of  data  are  calibrated  to  the  same  reference  time  (engine 
urs) .  Next  the  values  for  the  data  vector  are  calculated  for 
e  date  requested, and  finally  the  data  vector  is  checked  for 
normalities. 


FIGUKE  -  8 


ENGINE  SERIAL  NUMBER:  701264  AIRCRAFT  SERIAL  NUMBER:  160396 
INCIDENT  DATE:  62090  3/31/62 

REMOVED  PART  NUMBER: 

INSTALLED  PART  NUMBER: 

WORK  UNIT  HEADING:  IGNITION  SYSTEM 

•••WARNING***  WORK  UNIT  SUBHEAOING  NOt  FOUND 

NORK  UNIT  TITLE:  EXCITER  INPUT  CABLE 

WORK  UNIT  CODE:  23BA700 

HO W  MALFUNCTIONED  CCOE:  020 

HOW  MALFUNCTIONED  CODE:  020 

HOW  MALFUNCTIONED  CODE:  000 

ACTION  TAKEN  CODE:  B 

ACTION  TAKEN  CCOE:  B  ' 

ACTION  TAKEN  CCOE:  0 
MIEN  DISCOVERED  COOE:  H 
WHEN  DISCOVERED  CCOE:  H 
MAN  HOURS:  000005 


ENGINE  SERIAL  NUMBER:  701430  AIRCRAFT  SERIAL  NUMBER:  701430 
INCIOENT  DATE:  82064  3/  5/62 

REMOVED  PART  NUMBER:  ■ 

INSTALLED  PART  NUMBER:  754495 

NORK  UNIT  HEADING:  EXHAUST  SECTION 

MCRK  UNIT  SL'BHEAOING:  AFTERBURNER  DIFFUSER  ASSEMBLY 

NORK  UNIT  TITLE:  AFTERBURNER  DIFFUSER  ASSEMBLY 

NORK  UNIT  CCOE:  23B4400 

HON  MALFUNCTIONED  CODE:  799 

HON  MALFUNCTIONED  CCOE:  799 

HON  MALFUNCTIONED  CODE:  000 

ACTION  TAKEN  COOE:  Q 

ACTION  TAKEN  CODE:  Q 

ACTION  TAKEN  COOE:  P 

NHEN  DISCOVERED  COOE:  0 

WHEN  DISCOVERED  COOE:  0 

MAN  HOURS:  000003 


INCIDENT  OATE:  82068  3/  9/62 

•••WARNING"**  AIRCRAFT  SERIAL  NUMBER  CHANGEO  AND  THE  ENGINE  SERIAL  HUMBER  DIDN'T 
ENGINE  SERIAL  KISSER  =  701430  AIRCRAFT  SERIAL  NUMBER  =  160899 
REMOVED  PART  NUMBER: 

INSTALLED  PART  NUMBER: 

•••WARNING****  WORK  UNIT  HEAOING  NOT  FOU18) 

•••WARNING**#  WORK  UNIT  SU8HEA0ING  NOT  FOUND 

WORK  UNIT  TITLE:  IGNITION  SYSTEM 

MCRK  UNIT  CCDE:  2ID0000 

HOW  MALFUNCTIONED  COOE:  600 

HOW  MALFUNCTIONED  CODE:  800 

HOW  MALFUNCTIONED  CODE: 

ACTION  TAKEN  COOE:  0 
ACTION  TAKEN  COOE:  Q 
ACTION  TAKEN  CCOE: 

WHEN  DISCOVERED  COOE:  0 
NHEN  DISCOVERED  CCOE:  0 
MAN  HOURS:  000011 
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The  calibration  of  the  data  sources  is  done  with  respect 
to  the  engine  hours  since  overhaul  for  the  NOAP,  test  cell, 
and  the  3M  data.  In  order  to  calculate  the  engine  hours  of  a 
particular  3M  or  test  cell  data  entry,  the  date  is  used  to 
interprets te  an  estimate  of  engine  hours. 

The  calculation  of  the  data  vector  points  varies  from 
point  to  point.  For  points  1  through  127,  the  test  cell  data 
is  used  as  described  in  Section  3.2.  Points  128  through  271 
involve  linearly  interpolating  between  available  wear  metal 
contractions  in  order  to  obtain  the  concentration  at  the 
observation  date.  To  keep  this  interpolation  from  causing  un¬ 
reasonable  results  through  large  extrapolations,  average  values 
of  the  wear  metal  concentrations  at  test  cells  have  been  used. 

If  no  test  cell  data  was  available,  and  if  dates  latter  than  the 
lattest  available  data  are  requested,  the  lattest  concentrations 
available  are  used  instead  of  the  extrapolated  values.  Points 
272  through  275  are  interpreted  from  available  data.  The  3M 
data  Points  276  through  385  are  calculated  by  counting  the 
occurrences  of  the  required  codes.  In  the  cases  which  indicate 
a  number  of  hours  should  be  counted  back,  the  date  corresponding 
to  that  number  of  hours  in  the  past  is  calculated  through  inter¬ 
polation  and  becomes  the  starting  point  for  the  counts.  The 
roost  active  codes  were  calculated  using  the  initial  data  received. 
This  data  was  limited  in  the  number  of  cases  it  contained  and  in 
future  study  these  most  active  codes  should  be  recalculated  on  a 
larger  data  set. 


In  the  process  of  this  study,  a  possible  problem  with  the 
interpolation  used  was  encountered  which  should  be  addressed 
in  any  following  studies.  The  problem  occurs  in  the  following 
situation;  if  an  engine  failure  is  observed  between  NOAP  samples 
the  oil  concentrations  corresponding  to  the  time  the  engine  of 
the  failure  will  be  obtained  by  interpolation  between  dates  that 
the  wear  metal  concentrations  indicated  a  good  engine  and  dates 
where  an  engine  failure  was  indicated. 


The  software  scan  of  oil  concentrations  is  designed  to 
alert  the  user  of  any  values  that  fall  outside  a  normally 
acceptable  range.  This  was  designed  to  catch  values  that 
were  extrapolated  without  bound.  However,  as  described  earlier, 
that  condition  was  eliminated.  The  test  still  is  very  useful 
in  searching  for  possible  bad  data.  A  final  scan  for  errors 
is  done  on  the  test  cell  date.  The  test  cell  date  from  the 
NOAP  and  the  TEST  CELL  data  is  compared  and  a  warning  is  printed 
if  they  do  not  match  (an  example  can  be  seen  in  Figure  9) . 

When  the  test  cell  dates  do  not  match  the  date  on  the  NOAP  data, 

data^as  iS  USef  !inCS  lt  iS  assurned  that  the  NOAP  date  and 

Dhase^AB  5  6d  by  the  US6r  in  the  manual  second  pass 

phase  as  described  in  Section  3.3.  * 
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4.0  DEVELOPMENT  ANALYSIS  AND  EXPECTED  PERFORMANCE  OF 
FAILURE  DETECTION  PROCEDURES 

This  section  will  describe  how  the  data  vectors  will 
be  processed  to  derive  the  algorithms  and  other  analysis 

tools.  This  will  be  followed  by  an  analysis  _ 

of  the  performance  and  other  characteristics  of  the  most 
significant  algorithms  which  were  derived  during  this  study. 

4.1  ADAPT  APPROACH  AND  ANALYSIS  TOOLS 

The  ADAPT  approach  to  empirical  analysis  which  was  used 
to  derive  and  test  the  algorithms  for  this  study  was  originally 
developed  at  AVCO  Corporation  and  called  the  AVCO  Data  Analysis 
and  Prediction  Techniques  (ADAPT) .  This  technology  has  been 
acquired,  further  developed  and  applied  by  the  ADAPT  Service 
Corporation.  The  approach  has  been  discussed  in  the  literature1' 
It  is  also  presented  in  detail  in  Appendices  A-C.  However, 
the  key  components  of  the  method  will  be  repeated  here. 

The  general  concept  of  the  ADAPT  approach  is  to  take  the 
data  set  from  the  original  high  dimensional  measurement  space 
and  transform  it  to  a  lower  dimensional  but  optimal  feature 
or  analysis  space,  unlike  roost  pattern  recognition  approaches, 
the  features  are  derived  objectively  and  are  the  eigenvectors 
or  optimal  empirical  orthogonal  functions  associated  with  the 
training  set.  Figure  10  illustrates  the  steps  in  this  approach. 
The  data  vector s^  constructed  as  described  in  Section  3  are 
represented  by  the  schematic  data  vector  shown  in  the  upper 
left  hand  corner  of  this  diagram.  These  data  vectors  are 
transformed  into  an  ordered  optimal  coordinate  system  (i.e., 
feature  space  )  defined  by  the  Karhunen-Loeve  expansion  .  In 
addition  to  being  objectively  determined,  these  features  are 
complete  in  the  sense  that  before  truncation  all  of  the  in¬ 
formation  in  the  original  data  space  is  retained.  The  truncation 
is  based  on  analysis  of  the  explained  variation  rather  than  the 
usual  arbitrary  rejection  or  selection  of  features. 


(1)  Shenk,  William  E.  et  al,  "The  Estimation  of  Extratropical 
Cyclone  Parameters",  J.Appl.  Meteor.  12,  pp  441-451,  1973. 

(2)  Hunter,  H.E.,  et  al,  "An  Objective  Method  for  Forecasting 
Tropical  Cyclone  Intensity  Using  NIMBUS  5  Electrically  Scan¬ 
ning  Microwave  Radiometer",  J.  Appl.  Meteor.  20,  pp.  137-145 
1981. 

(3)  Andrews,  H.C. ;  Introduction  to  Mathematical  Techniques  in 
Pattern  Recognition.  Wiley,  1972. 


FIGURE  -  10 

OVERVIEW  OF  ADAPT  ANALYSIS  TECHNIQUES  USED 


Although  the  eigenvector  or  Karhunen-Loeve  expansion 
are  well  known  in  the  literature,  there  are  a  number  of 
problems  which  have  plagued  the  application  of  this  technique 
to  real  data  problems  such  as  the  present  problem.  The  ADAPT 
approach  to  deriving  the  eigenvectors  has  overcomed  these 
difficulties.  A  detail  description  of  these  problems  and 
how  they  are  overcome  using  the  ADAPT  approach  to  the  deriva¬ 
tion  is  presented  in  Appendix  B. 

The  analysis  and  derivation  of  classification  or  regression 
algorithms  is  performed  in  the  feature  or  optimal  analysis 
space.  This  is  indicated  by  the  right  hand  side  of  Figure  10. 
Both  the  regression  and  pattern  recognition  algorithms  are 
derived  using  the  ADAPT  developed  independent  eigenscreening 
approach.  This  approach  differs  from  conventional  screening 
algorithm  developments  in  two  ways:  1)  the  screening  as 
performed  in  eigenvector  space  and  2)  the  decision  as  to 
whether  a  given  direction  should  be  retained  or  rejected  is 
based  on  independent  ("U"  method  of  Eackenbrach,  Mickey4) 
rather  than  dependent  test  results.  This  last  difference  is 
of  critical  importance  since  it  can  be  shown  that  for  most 
problems  the  use  of  dependent  testing  gives  incorrect  conclu¬ 
sions  regarding  the  usefulness  of  a  feature.  Although  this 
conclusion  is  generally  accepted  throughout  the  field  with 
respect  to  performance  estimates,  it  is  ignored  in  almost  all 
screening  procedures.  The  details  of  the  ADAPT  independent 
eigenscreening  approach  are  presented  in  Appendix  C. 

After  the  algorithms  have  been  derived,  they  may  be  trans¬ 
formed  back  to  the  original  measurement  space  for  further 
analysis.  This  is  possible  since  the  eigenvector  transformation 
is  orthogonal  and,  therefore,  its  inverse  is  always  known. 

When  the  algorithm  has  been  transformed  back  to  the  original 
measurement  space,  we  call  it  a  relative  importance  vector 
because  it  defines  the  importance  of  each  of  the  original 
variables  to  the  decision  which  is  being  made  by  the  algorithm. 


(4)  Lackenbrach,  P.A.  and  Mickey,  M.R. ;  "Estimation  of  Error 

Rates  in  Discriminant  Analysis",  Technometrics,  10,  pp  11-17, 
1968. 
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This  brief  review  of  the  ADAPT  approach  allows  us  to 
define  two  important  analysis  tools  which  will  be  made 
available  as  part  of  the  menus  if  the  recommended  incipient 
failure  detection  demonstration  system  is  implemented.  These 
tools  are  the  scatter  plot  projection  on  the  significant 
eigendirections  and  the  relative  importance  vectors  associated 
with  an  unusual  case.  These  tools  would  be  made  available 
to  experienced  analysts  and  would  not  be  the  primary  means 
for  defining  and  diagnosing  incipient  engine  failures.  The 
relative  importance  vector  is  defined  in  Appendix  A.  In  the 
analysis  part  of  the  demonstration  system,  algorithms  may  be 
derived  separating  any  given  data  vector  from  an  ensemble 
of  the  other  data  vectors  in  the  system.  The  relative  importance 
vector  associated  with  this  algorithm  would  define  which  of 
the  measurements  are  responsible  for  the  unhealthy  or  unusual 
behavior  associated  with  the  engine  in  question. 

The  description  of  the  ADAPT  feature  or  eigenvector 
space  points  out  that  this  is  an  optimal  space  in  the 
sense  that  the  greatest  amount  of  information  is  presented 
in  the  minimum  amount  of  numbers.  It  follows  that  scatter 
plot  projections  of  this  information  on  these  eigendirections 
are  ideally  suited  for  human  investigators  to  search  for 
clusters.  Usually,  this  is  only  justified  for  the  dominant 
eigendirections  which  because  of  the  nature  of  the  eigen¬ 
vector  expansion  are  both  unique  and  efficient  in  presenting 
the  inter-relationships  between  the  various  cases.  Thus, 
a  relatively  small  number  of  two  dimensional  scatter  plots 
often  presents  the  essence  of  the  information  contained 
in  an  entire  data  set.  By  comparing  the  track  of  an  engine 
on  such  a  scatter  plot  with  itself  or  other  engines,  a 
great  deal  can  be  learned  by  the  experienced  investigator. 

In  the  following  sub  sections,  we  will  present  examples 
of  these  presentations  and  how  they  may  be  used  by  an  experienced 
analyst  to  improve  the  diagnostics  of  an  engine  failure.  How¬ 
ever,  we  must  emphasize  all  of  the  performances  which  have  been 
given  and  which  will  be  given  are  based  on  the  automated 
algorithms  and  do  not  include  any  additional  gain  that  could 
be  obtained  by  an  experienced  analyst  using  these  tools  which  will 
be  made  available.  We  believe  that  these  tools  will  be  useful 
in  the  same  sense  that  the  presentation  of  the  raw  data  has  been 
found  useful  in  the  present  NOAP  system. 


Engine  Health  Eigenvector  Expansion 


The  data  vectors  were  prepared  as  described  in  Section  3 
for  the  590  observations  or  samples  taken  from  the  14  engines 
listed  in  Table  2.  The  first  column  of  Table  2  identifies 
the  engine  number,  the  second  column  identifies  the  sample 
or  observation  number  and  the  symbol  which  will  be  used  for 
that  group  of  sanples  in  the  scatter  plot  presentation. 

This  will  be  discussed  later  in  Section  4.2.  The  third 
column  presents  the  ID  of  the  sample.  F  stands  for  a  failure, 
G  for  a  good  engine  and  X  for  a  questionable  engine.  These 
will  also  be  discussed  in  more  detail  in  later  sections. 

The  fourth  and  fifth  columns  describe  the  failure  if  any  and 
present  other  information  regarding  the  engines. 


The  eigenvector  transformation  used  to  transform  the  data 
from  the  measurement  space  to  the  optimal  analysis  or  feature 
space  was  derived  using  480  of  the  observations  listed  in 
Table  2.  One  hundred  and  ten  of  the  observaions  listed  in 
Table  2  were  not  used.  These  110  observations  were  deleted 
to  provide  a  better  balance  between  a  number  of  observations 
for  each  of  the  engines  used  and  also  a  better  balance  between 
good  and  incipient  failure  observations.  It  is  important  to 
maintain  approximately  equal  representations  of  the  known  major 
divisions  in  constructing  the  eigenvector  transformation  if 
one  desires  that  the  resulting  transformation  represent  these 
known  variations  equally. 

The  average  of  the  480  data  vectors  used  to  derive 
the  eigenvectors  was  constructed.  This  average  is  presented 
in  Figure  11.  Since  all  of  these  data  vectors  were  equalized 
such  that  each  of  their  components  have  a  value  between  one 
and  two,  the  mean  of  each  component  is  1.5.  Examination 
of  Figure  11  shows  that  the  test  cell  measurements  have 
their  mean  approximately  equal  to  their  average.  The  oil 
measurements  in  general  have  their  mean  significantly  lower 
than  the  average.  This  suggests  that,  in  general,  there 
are  a  large  number  of  outlier  cases  occurring  in  the  NOAP 
data.  The  number  of  outlier  cases  is  even  larger  for  the 
3M  data  where  the  average  is  much  smaller  than  the  mean. 


Before  deriving  the  eigenvector  transformations,  the  data 
set  used  to  derive  the  eigenvectors  was  given  a  zero  mean 
by  subtracting  the  average  data  vector  shown  in  Figure  11  for 
each  of  the  data  histories.  This  was  done  to  all  590  observations 
listed  in  Table  2  because  the  zero  mean  data  vectors  would  also 
be  desired  for  all  of  the  algorithm  development  which  will 
follow. 
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TABU  2  -  SUMMARY  OT  CASES  USED 


EMC.  MO. 

SAMPU  MOS.(SYM) 

ID 

TRUTH  DATA 

REMARKS 

687021 

1-15  (Z) 

XI 

FUBL  CONTROL  FAIL  •  TEST  CELL 

1)1<I  YR'S  DATA 

16-45  (1) 

01 

QOOD 

46-60  (Y) 

PI 

FAN  INLET  CASE  CRACKED,  COWUS- 
TIOM  C HAMER  OUTER  DUCT  CRACKED, 
BAD  MAIM  FUEL  SPRAY  PATTERN  6 
EXCESSIVE  PT7 

687279 

61-70  (X) 

F2 

CRACKED  6  BENT  AB  FUEL  PUMP 

1)3  YR'S  DATA 

71-80  (2) 

02 

2) USED  F3  FOR 

81-95  (W) 

P3 

CLOGGED  SUMP  STRAINER  IN  MAIN 
OIL  PUMP 

Tim  TO  FAIL 

96-145 (V) 

P4 

DIRTY  (MOM- METALLIC)  OIL 

ALGORITHM 

146-215(3) 

03 

3)F4  PICKED 

216-22S(U) 

P5 

MAIM  FUBL  SYSTEM  6  AB 

IGNITER  FUEL  VALVE 

BY  NOAP 

679390 

226-250(4) 

04 

679412 

251-270(5) 

OS 

695024 

271-300 (T) 

F6 

MAIM  OIL  FILTER  LOOSE  A  OUT 

OF  ADJUSTMENT!  OIL  STRAINER 

301-310(8} 

F7 

A.B.  FUBL  SYSTEM  FAILURE 

695195 

311-330(6) 

06 

695489 

331-360 (R) 

F8 

ENGINE  FAILED  TO  OPERATE  & 

HAD  OIL  LEAK 

701161 

361-380(7) 

G7 

701251 

381-420(8) 

08 

701251A 

421-440(0) 

X2 

HYBRED  CASE 

GOOD  CELL,  BAD  OIL 

7012 SIB 

441-460 (P) 

F9 

failed  test  cell  performance 

PASS* (AB  FUEL  CONTROL  FAILED 

'ADJUSTED  TO 

IM  2-WEEKS 

701255 

461-480(9) 

G9 

701257 

481-505(0) 

F10 

EXCESSIVE  VIBRATION 

506-550 (N) 

Fll 

UNKNOWN  FAILURB-BaSED  ON  RETURN  FOR 

OVERHAUL-NO  FAILURE  DOCUMENTATION 

701264 

551-565 (M) 

F12 

IGNITION  SYSTEM 

701430 

566-575(a) 

010 

701434 

576-590(B) 

Cll 
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FIGURE  -  11 


AVERAGE  OF  480  ENGINE  HEALTH 
MEASUREMENTS  USED  TO  DERIVE  EIGENVECTORS 


Test  Cell 


NOAP 


The  results  o£  the  ADAPT  eigenvector  transformation 
derivation  program  are  summarized  in  Figure  12.  This  figure 
presents  a  plot  of  the  percent  of  the  information  contained 
in  the  entire  data  set  which  is  represented  by  each  of  the 
eigendirections  or  features  of  the  optimal  analysis  space. 

This  figure  shows  two  curves.  The  upper  curve  is  the  cumula¬ 
tive  sum  of  the  lower  curve.  Thus,  entering  this  figure  at 
an  abscissa  value  of  1,  we  find  both  the  upper  and  lower 
curves  have  an  ordinate  value  of approximately  25%.  This 
indicates  that  the  first  eigendirection  or  feature  explains 
approximately  25%  of  the  variation  or  in  more  intuitive 
terms  contains  25%  of  the  information  which  was  contained  in 
the  entire  data  set  of  480  observations  of  385  measurements. 
Entering  at  an  abcissa  value  of  2,  the  lower  curve  has  a 
value  of  approximately  15%  and  the  upper  curve  a  value  of 
approximately  40%.  This  indicates  that  a  second  eigendirec¬ 
tion  contains  15%  of  the  information  in  the  total  data  set 
and  that  the  first  and  second  eigendirections  taken  together 
contain  40%  of  the  information.  Thus,  if  one  were  to  plot  a 
scatter  plot  of  the  first  and  second  eigendirections  showing 
each  of  the  observations  on  the  first  and  second  eigendirections 
this  single  plot  would  contain  40%  of  the  information  which 
could  be  obtained  by  examining  all  of  385  measurements  for 
the  480  cases.  This  demonstrates  the  efficiency  of  the  ADAPT 
scatter  plots  for  enabling  the  human  to  observe  natural 
clusterings  of  the  data.  Considerable  additional  information 
regarding  this  eigenvector  expansion  including  plots  of  the 
more  significant  eigenvectors  and  scatter  plots  on  a  number 
of  these  eigenvectors  are  included  in  Appendix  D  to  this  report. 

The  scatter  plot  of  the  data  on  the  first  two  eigen¬ 
directions  is  shown  in  Figure  13.  This  figure  shows  considerable 
grouping  of  the  cases.  Much  of  this  grouping  is  grouping  by 
engine.  This  suggests  that  increasing  the  number  of  engines 
will  provide  a  more  complete  definition  of  the  variation  and 
should  Improve  the  results  beyond  those  presented  in  this 
report.  The  symbols  used  on  Figure  13  are  those  listed  in 
Column  2  of  Table  2.  The  numerals  and  letters  A  and  B  indicate 
good  engines  and  the  remaining  letters  indicate  bad  engines. 

Thus,  we  see  that  although  there  is  considerable  grouping  in 
these  first  two  eigendirections,  it  is  not  predominantly  good 
versus  bad  engine. 


Engine  Serial  No.  701257  clusters  in  a  group  removed 
from  the  rest.  This  engine  was  the  only  engine  used  for 
which  there  was  no  3M  data  and  this  is  probably  why  it  is 
separated  from  the  rest.  The  two  engines  covering  relative 
large  regions  are  enclosed  in  solid  contours.  These  three 
engines  are  the  two  engines  for  which  data  is  available  for 
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FIGURE  -  12 

INFORMATION  (IE  EXPLAINED  VARIATION)  AS  A  FUNCTION  OF 
DIMENSIONALITY  FOR  THE  EIGENVECTOR  REPRESENTATION  OF  THE  ENGINE  HEALTH  DATA  VECTOR 
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FIGURE  —  13 

PROJECTION  ON  FIRST  AND 
SECOND  ENGINE  HEALTH  E I GEND I RECT I  ON 


longest  period  of  time.  This  suggests  that  if  data  is 
liable  for  more  engines  over  the  entire  space  between  two 
cessive  overhauls,  the  performance  of  the  algorithms  derived 
m  this  data  should  improve. 

DETECTION  OF  INCIPIENT  FAILURES 

This  section  presents  information  which  defines  in  detail  the 
srformance  of  the  automated  algorithms  for  detection  of 
tcipient  engine  failures  as  well  as  an  analysis  tool  which 
m  be  used  by  an  experienced  analyst  to  further  understand 
te  health  of  an  engine.  We  shall  also  present  the  relative 
iportance  vectors  associated  with  each  of  the  algorithms 
>  illustrate  which  measurements  are  important  to  the  decisions. 

The  performance  of  the  automated  failure  detection  algorithms 
in  be  described  either  in  terms  of  a  probability  of  error  or 
i  terms  of  a  plot  of  the  detection  statistic  for  each  of 
ie  590  cases  used  in  this  study.  Both  the  probability  of 
ror  and  the  detection  statistics  presented  in  this  and  follow- 
ig  sections  are  derived  using  the  independent  test  method  U 
iscribed  by  Lackenbrach  and  Mickey4. 

itection  Algorithm  Incipient  Engine  Failures 

The  first  algorithm  which  we  shall  present  is  the  algorithm 
•r  the  detection  of  incipient  engine  failures.  This  algorithm 
s  developed  by  separating  those  observations  indicated  as 
ilures  in  Table  2  from  the  cases  indicated  as  good  engines, 
ie  probability  of  error  for  this  algorithm  was  0.14.  Figure 
presents  the  detection  statistic  for  each  of  the  590 
servations  used  in  this  study  after  application  of  this 
gorithm. 

The  detection  statistic  is  the  quantity  which  is  calculated 
the  algorithm  and  is  used  to  decide  whether  a  failure  will 
cur.  That  is,  after  one  calculates  the  detection  statistic 
om  the  algorithm,  it  is  compared  with  the  threshold  and 
the  value  exceeds  the  threshold,  we  predict  that  incipient 
ilure  will  occur.  By  setting  the  thresf  old,  one  may  adjust 
e  detection  probability  and  false  alarm  ratio.  In  the 
velopment  of  the  final  algorithms  to  be  used  in  the  demonstra- 
on  system,  this  will  be  carried  out  after  discussions  with 
vy  personnel  to  insure  the  best  trade-off  between  false 
arms  and  detections. 

We  can  get  a  good  understanding  of  the  performance  of 
ese  algorithms  simply  by  examining  a  plot  of  their  detection 
a ti sties  for  each  of  the  cases.  The  threshold  for  the  plot 
own  in  Figure  14  is  at  a  detection  statistic  value  of  0. 
us,  any  positive  detection  statistic  value  indicates  a  bad 
gine,  a  negative  value  indicates  a  good  engine.  The  ordinant 


35 


-60.00  -HO. 00  -20.00  0.00  20.00  40.00 


FIGURE  -  14 


DETECTION  STATISTIC  FOR 
DETECTION  OF  INCIPIENT  ENGINE  FAILURES 


0.0 
10.00 


100.0  200.0  300.0  400.0 

HOL  NUMBER 


500.0  600.0  700.0 

16  SEP  198? 


36 


^1 


r„  • 

>  ■ 


N 


i' 

. ' 

i 

1 « 

r 


of  Figure  14  is  the  detection  statistic,  the  abscissa 
indicated  as  HOL  number  is  the  observation  or  sample  for 
which  the  detection  statistic  is  calculated.  This  plot  is 
actually  a  plot  of  a  great  many  individual  bars.  Because 
of  the  large  number  of  cases  presented  in  a  small  area,  they 
have  run  together  and  appear  as  shaded  area  rather  than  as 
individual  bars  on  the  bar  chart.  Figure  15  presents  a 
plot  of  the  first  100  cases  of  Figure  14.  The  purpose 
of  this  presentation  is  to  illustrate  that  although  on 
Figure  14,  the  data  looks  like  solid  blobs,  these  are  really 
made  up  of  individual  bars  representing  detection  statistic 
occurring  for  a  given  observation  or  HOL  number  of  the  engine. 

In  order  to  keep  the  presentation  on  a  single  sheet  so  that 
the  reader  may  get  the  overall  picture,  we  have  had  to  allow 
these  bars  to  run  together,  however,  comparison  of  Figures  14 
and  15  should  make  clear  the  meaning  of  the  information 
presented  in  Figure  14  and  subsequent  detection  statistic 
plots  presented  in  this  report. 

To  assist  in  the  interpretation  of  Figure  14,  two 
additional  rows  of  information  have  been  added.  These  are: 

1)  upward  and  2)  downward  arrows  indicating  engines  and 

failures,  respectively.  The  downward  facing  arrows  on  the 

upper  portion  of  this  plot  indicate  the  occurrence  of  each 

of  the  failures  defined  in  Table  2.  Each  of  these  downward 

arrows  are  identified  by  the  ID  symbol  and  number  (if  applicable) 

shown  in  Table  2.  Thus,  the  occurrence  of  the  failures  are 

defined  by  these  downward  pointing  arrows.  Those  arrows  for 

which  the  data  preceding  them  at  some  point  has  positive 

detection  statistics  and,  therefore,  for  which  this  algorithm 

would  successfully  identify  the  occurrence  of  the  failure  has 

been  indicated  by  a  solid  downward  pointing  triangle  above  the 

appropriate  ID.  The  ID  included  in  a  circle,  F4,  is  the  single 

case  in  the  present  data  set  which  was  called  by  the  present  operations 

oil  analysis.  The  two  failure  ID's,  F9  and  F10,  enclosed  in 

squares  were  the  two  cases  called  in  the  test  cell. 

In  the  lower  half  of  the  figure,  there  are  a  series  of 
upward  pointing  arrows.  These  arrows  indicate  the  beginning 
of  the  observations  of  a  new  engine.  Thus,  there  is  an  upper 
arrow  occurring  at  a  HOL  number  of  one  which  is  the  beginning 
data  on  the  first  engine.  Referring  to  Table  2,  we  see  that 
this  is  Engine  No.  687021.  The  next  upward  pointing  arrow 
occurs  at  HOL  Number  61  again  referring  to  Table  2  we  see  that 
this  is  the  beginning  of  Engine  687279. 
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FIGURE  15  DETECTION  STATISTIC  FOR 
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Detail  examination  of  Figure  14  in  conjunction  with 
Table  2  not  only  shows  which  failures  have  and  have  not  been 
correctly  identified,  but  also  gives  an  indication  of  how  long 
in  advance  of  the  failure  one  can  first  see  the  occurrence  of 
the  failure  and  hew  strongly  it  has  been  detected.  One  caution 
in  interpreting  Figure  14  is  that  interpolation  (see  Section  3.5) 
was  used  between  oil  samples.  Thus,  if  a  failure  has  occurred 
and  been  corrected  and  the  engine  does  not  change,  the  oil 
samples  immediately  following  that  failure  are  interpolations 
between  the  most  recent  oil  sample  occurring  prior  to  the 
failure  and  the  nearest  oil  sample  after  the  failure.  Thus, 
we  often  see  a  gradual  slope  to  the  right  of  a  failure  occurring 
within  a  given  engine.  This  is  due  to  the  interpolation  pro¬ 
cedures  and  clearly  would  not  occur  in  the  real  world  where  the 
algorithm  would  normally  only  be  applied  at  the  time  the  sampling 
was  accomplished. 

Figure  14  also  shows  one  case  identified  as  a  failure  for 
which  a  failure  was  not  recorded  in  the  3M  or  NOAP  data.  This 
is  ID  number  G7  on  Table  2  and  is  also  indicated  by  a  triangle 
on  Figure  14.  Although  we  might  attribute  this  to  a  false 
detection  by  the  algorithms  or  an  actual  failure  which  was 
observed,  but  not  recorded  in  the  3M  system,  there  is  some 
evidence  that  it  was  an  incipient  failure  which  "got  well" 
and  therefore  was  never  observed.  This  evidence  is  the  negative 
slope  of  the  detection  statistic.  We  have  pointed  out  that 
negative  slopes  can  occur  after  a  failure  because  of  the  inter¬ 
polation  used  for  this  study.  However,  this  is  only  possible 
when  the  data  preceding  the  negative  slope  represented  a  failure 
of  the  same  engine.  However,  G7  occurred  very  shortly  after 
the  engine  left  the  test  cell,  which  in  the  context  of  this 
study  represents  the  "birth  of  the  engine"  and  by  definition 
represents  a  healthy  engine.  Thus,  the  negative  slope  of  G7 
is  not  a  result  of  the  interpolation! 

The  only  other  explanation  for  a  negative  slope  is  that 
the  engine  is  "getting  well*  as  it  gets  older.  An  example  of 
this  may  be  oil  contaminated  with  sufficiently  large  particles 
that  the  oil  filter  slowly  removes  them.  In  this  example,  the 
initial  reading  may  indicate  a  unhealthy  engine,  but  succeeding 
samples  would  be  progressively  more  healthy,  or  the  slope  of 
the  detection  statistic  would  be  negative.  Thus,  by  the  process 
of  elimination,  we  believe  the  G7  was  initially  a  valid  case  of 
an  unhealthy  engine  which  gradually  got  well  and  was,  therefore, 
never  discovered.  Although,  if  it  had  been  discovered  and 
corrected,  the  engine  may  have  lasted  longer  before  requiring 
an  overhaul. 

Table  3  repeats  the  information  plotted  in  figure  14 
and  15  in  tabular  form.  The  values  in  table  3  are  7.5  smaller 
than  those  in  the  figure  this  adjustment  of  the  data  was  made 
so  that  the  seperation  would  be  easier  to  see. 
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TABLE  3 

DETECTION  STATISTIC  FOR  DETECTION  OF  INCIPIENT  ENGINE  FAILURES 

CONTINUED 

HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


201  -0. 1682596E  02 
205  -0.1679286E  02 
209  -0.1675974E  02 
213  -0. 1672661E  02 
217  -0.7641908E  01 
221  -0.7542549E  01 
225  -0.7443193E  01 
229  -0.4588925E  02 
233  -0. 443904 9E  02 
237  -0.42B9169E  02 
241  -0.3682193E  02 
245  -0.3669029E  02 
249  -0.3671257E  02 
253  -0.2457710E  02 
257  -0.2449353E  02 
261  -0.2659567E  02 
265  -0.2927513E  02 
269  -0.2B70296E  02 
273  0.5403949E  01 
277  -0.2740580E  01 
281  -0.2894591E  01 
285  -0.2843789E  01 
289  -0.2176266E  01 
293  -0. 52786 35E  00 
297  -0.1603148E  01 
301  0.3303407E  01 
305  0.1698288E  01 
309  -0.2485Z47E  00 
313  -0.9064890E  01 
317  -0.8872767E  01 
321  -0.9474847E  01 
325  -0.1891789E  02 
329  -0.1872310E  02 
333  0.5154556E  00 
337  -0 . 1403463E  00 
341  -0.6813905E  00 
345  -0.6441395E  00 
349  -0.3262650E  00 
353  -0.8354366E-02 
357  0.2636824E  00 
361  -0.42090S4E  00 
365  -0.3013947E  01 
369  -0.4192301E  01 
373  -0.5370766E  01 
377  -0.6392705E  01 
381  -0.2377878E  02 
385  -0.2309483E  02 
389  -0.2193874E  02 
393  -0.2078261E  02 
397  -0. 1962650E  02 


202  -0. 1681769E  02 
206  -0.1678458E  02 
210  -0. 1675145E  02 
214  -0.1671834E  02 
218  -0.7617068E  01 
222  -0. 7517712E  01 
226  -0. 46872 10E  02 
230  -0.4551456E  02 
234  -0.4401575E  02 
238  -0.4251700E  02 
242  -0.3678900E  02 
246  -0.3665736E  02 
250  -0.3673105E  02 
254  -0.2455621E  02 
258  -0.2447266E  02 
262  -0.2918416E  02 
266  -0.2872351E  02 
270  -0.2875874E  02 
274  0.5321344E  01 
278  -0.2779077E  01 
282  -0.2933111E  01 
286  -0.2676909E  01 
290  -0. 20666 18E  01 
294  -0.7897702E  00 
298  -0. 1246010E  01 
302  0.3319881E  01 
306  0. 1661471E  01 
310  -0.2853956E  08 
314  -0.9211164E  01 
318  -0.9035929E  01 
322  -0.9621119E  01 
326  -0.1886919E  02 
330  -0.1867441E  02 
334  0.3514962E  00 
338  -0.3042976E  00 
342  -0.7141770E  00 
346  -0.5646822E  00 
350  -0.2467876E  00 
354  0. 71136 77E-01 
358  0.2824440E  00 
362  -0.1068998E  01 
366  -0.3308489E  01 
370  -0.4486950E  01 
374  -0.5665446E  01 
378  -0.6212405E  01 
382  -0.2370918E  02 
386  -0.2280S80E  02 
390  -0. 2164970E  02 
394  -0.2049359E  02 
398  -0. 1077949E  02 


203  -0. 1680942E  02 
207  -0. 1677629E  02 
211  -0. 1674316E  02 
215  -0. 1671005E  02 
219  -0. 7S92230E  01 
223  -0.7492872E  01 
227  -0.4663866E  02 
231  -0.4513986E  02 
235  -0.4364107E  02 
239  -0.4214226E  02 
243  -0.36 75613E  02 
247  -0 . 3667575E  02 
251  -0.2482631E  02 
255  -0.2453532E  02 
259  -0. 24451 75E  02 
263  -0.2925125E  02 
267  -0. 28831 39E  02 
271  0.5569159E  01 
275  0.5238733E  01 
279  -0.2817589E  01 
283  -0.2971612E  01 
287  -0.2510048E  01 
291  -0. 1966812E  01 
295  -0. 1051687E  01 
299  -0.7768197E  00 
303  0.3058112E  01 
307  0. 1624660E  01 
311  -0.8772315E  01 
315  -0.9357464E  01 
319  -0.9182246E  01 
323  -0 . 9767396E  01 
327  -0. 1882047E  02 
331  0 . 99B7539E  00 
335  0. 187546 9E  00 
339  -0.4682544E  00 
343  -0.7469618E  00 
347  -0. 48520 16E  00 
351  -0.1673035E  00 
355  0. 1568396E  00 
359  0.3012075E  00 
363  -0 . 1717080E  01 
367  -0.3603130E  01 
371  -0.4781600E  01 
375  -0.5960039E  01 
379  -0.6032071E  01 
383  -0.2365236E  02 
387  -0.2251677E  02 
391  -0.2136067E  02 
395  -0.2020454E  02 
399  -0. 1049050E  02 


204  -0. 16801 13E  02 
208  -0.167680 IE  02 
212  -0. 1673489E  02 
216  -0. 7666744E  01 
220  -0.7567390E  01 
224  -0.7468034E  01 
228  -0.4626398E  02 
232  -0.4476517E  02 
236  -0.4326639E  02 
240  -0.4176764E  02 
244  -0.3672321E  02 
248  -0.3669415E  02 
252  -0.2470175E  02 
256  -0.2451443E  02 
260  -0.2655743E  02 
264  -0.2931828E  02 
268  -0.2880714E  02 
272  0.5486553E  01 
276  0.5156122E  01 
280  -0.2856088E  01 
284  -0.3010115E  01 
288  -0.2343162E  01 
292  -0.2649164E  00 
296  -0.1313611E  01 
300  -0.3699543E  00 
304  0.2378197E  01 
308  0. 1587843E  01 
312  -0.8918595E  01 
316  -0.1166625E  02 
320  -0 . 9328544E  01 
324  -0.9913679E  01 
328  -0. 1A77180E  02 
332  0.744U60E  00 
336  0.2359319E-01 
340  -0.6321981E  00 
344  -0.7236212E  00 
348  -0.4057366E  00 
352  -0.8783031E-01 
356  0.2449160E  00 
360  0.3199749E  00 
364  -0.2365159E  01 
368  -0.3897780E  01 
372  -0.5076128E  01 
376  -0.6254820E  01 
380  -0.5851728E  01 
384  -0.2338385E  02 
388  -0.2222775E  02 
392  -0.2107161E  02 
396  -0 . 1991550E  02 
400  -0. 1118836E  02 


TABLE  3 

DETECTION  STATISTIC  FOR  DETECTION  OF  INCIPIENT  ENGINE  FAILURES 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

1  -0.6075038E  01  2  -0.6037438E  01 

5  -0.5922445E  01  6  -0.5B86152E  01 

9  -0.5798734E  01  10  -0.5766377E  01 

13  -0.5669241E  01  14  -0.S636859E  01 

17  -0.1022254E  02  18  -0.1010676E  02 

21  -0.2131833E  02  22  -0.2049136E  02 

25  -0. 1508053E  02  28  -0.9322161E  01 

29  -0. 1386513E  02  30  -0.1547319E  02 

33  -0. 1471607E  02  34  -0.1329718E  02 

37  -0.1316225E  02  38  -0.9787459E  01 

41  -0.1347499E  02  42  -0.1217612E  02 

45  -0. 1326563E  02  48  -0.6356075E  01 

49  -0.4516901E  01  50  -0.3872260E  01 

53  -0. I546221E  01  54  -0.7117445E  00 

57  -0.3185608E  01  58  -0.3997791E  01 

81  -0.8282899E  01  82  -0.8274519E  01 

65  -0.8240193E  01  66  -0.8224023E  01 

69  -0.6093130E  01  70  -0.8034355E  01 

73  -0.1U7361E  01  74  -0.8467951E  00 

77  -0.3508854E-01  78  0.2355556E  00 

81  0.1490821E  01  82  0.1335173E  01 

85  0. 1097622E  01  86  0.3258515E  01 

89  0.5553464E  01  90  0.8468244E  01 

93  0.1448270E  02  94  0.1828297E  02 

97  0. 1253979E  02  98  0.1387488E  02 

101  0. 1282747E  02  102  0.1206238E  02 

105  0. 1263480E  02  106  0.9095583E  01 

109  0.4410972E  01  110  0.4352014E  01 

113  0.2122931E  01  114  0.1243500E  01 

117  -0. 1488571E  01  118  -Q.1940299E  01 

121  -0.3184000E  01  122  -0.3598581E  01 

125  -0.4143789E  01  126  -0.4232385E  01 

129  -0.4311930E  01  130  -0.3954283E  01 

133  -0.2881365E  01  134  -0.2523740E  01 

137  -0. 145082 3E  01  138  -0.1093201E  01 

141  -0.2028555E-01  142  0.3373380E  00 

145  0.U61077E  01  146  -0.2743499E  01 

149  -0.1712912E  02  150  -0.1702380E  02 

153  -0. 1642023E  02  154  -0.1571519E  02 

157  -0. 1652632E  02  158  -0.1645004E  02 

161  -0.1634544E  02  162  -0.1538455E  02 

165  -0. 1690974E  02  166  -0.1686995E  02 

169  -0.1697166E  02  170  -0.1602779E  02 

173  >0. 1852353E  02  174  -0.1855548E  02 

177  >0. 1865141E  02  178  -0.1868338E  02 

181  -0. 1695697E  02  182  -0.1763728E  02 

185  -0. 1712198E  02  186  -0.1695016E  02 

189  -0. 1692532E  02  190  -0.1691704E  02 

193  -0.1689220E  02  194  -0.1688391E  02 

197  •0.1685910E  02  198  -0.1685080E  02 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

3  -0.S999715E  01  4  -0.5961075E  01 

7  -0.5857450E  01  8  -0.5828748E  01 

11  -0.5733990E  01  12  -0.5701629E  01 

15  -0.5604485E  01  16  -0.1070250E  02 

19  -0.1056957E  02  20  -0.2199187E  02 

23  -0. 1889995E  02  24  -0.1749225E  02 

27  -0.7697547E  01  28  -0.1020233E  02 

31  -0. 1525981E  02  32  -0.1480079E  02 

35  -0.1455011E  02  36  -0.1501979E  02 

39  -0. 1323039E  02  40  -0.1404854E  02 

43  -0. 1129834E  02  44  -0.1166482E  02 

47  -0.5604072E  01  48  -0.5161559E  01 

51  -0.3215178E  01  52  -0.2380704E  01 

55  -0. 1000358E  01  56  -0.2373416E  01 

59  -0.3702036E  01  60  -0.3691925E  01 

63  -0.8266126E  01  64  -0.8256349E  01 

67  -0.8207871E  01  68  -0.8151897E  01 

71  -0.5937922E  01  72  -0.1387919E  01 

75  -0.5762320E  00  76  -0.3056574E  00 

79  0.3148680E  00  80  0.3941793E  00 

83  0. 1216483E  01  84  0.1113727E  01 

87  0. 1072967E  02  88  0.9177131E  01 

91  0.9639341E  01  92  0.U42081E  02 

95  0. 1982681E  02  96  0.8542032E  01 

99  0.1470341E  02  100  0.1243936E  02 

103  0. 1092713E  02  104  0.1167147E  02 

107  0.7485351E  01  108  0.5875163E  01 

111  0.4211942E  01  112  0.3002364E  01 

115  0.3640059E  00  116  -0.2723622E  00 

119  -0.2354864E  01  120  -0.2769429E  01 

123  -0.3966593E  01  124  -0.4055186E  01 

127  -0.4320971E  01  128  -0.4467684E  01 

131  -0.3596658E  01  132  -0.3239027E  01 

135  -0.2166099E  01  136  -0.1808452E  01 

139  -0.7355683E  00  140  -0.3779203E  00 

143  0.6949956E  00  144  0.8875434E  00 

147  -0.1733972E  02  148  -0.1723441E  02 

151  -0.1689340E  02  152  -0.1675047E  02 

155  -0.1637006E  02  156  -0.1660262E  02 

159  -0. 1639149E  02  160  -0.1636847E  02 

163  -0.1676987E  02  164  -0.1694954E  02 

167  -0.1683014E  02  168  -0.1690089E  02 

171  -0.1654175E  02  172  -0.1849149E  02 

175  -0.1858746E  02  176  -0.1861943E  02 

179  -0.1871535E  02  180  -0.1874731E  02 

183  -0.1746552E  02  184  -0.1729373E  02 

187  -0. 1694189E  02  188  -0.1693361E  02 

191  -0.1690877E  02  192  -0.1690048E  02 

195  -0.1687566E  02  196  -0.1686737E  02 

199  -0. 1684253E  02  200  -0.1683424E  02 


* 

I 
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TABLE  3 

DETECTION  STATISTIC  FOR  DETECTION  OF  INCIPIENT  ENGINE  FAILURES 


CONTItAJED 

HOL  DETECTION  STAT  HOL  DETECTION  STAT 

401  -0.1116935E  02  402  -0.1115034E  02 

40S  -0.9459343E  01  404  -0.9440324E  01 

409  •0.13B5412E  02  410  >0.1415839E  02 

413  -0. 1464174E  02  414  -0.1474848E  02 

417  -0. 1429339E  02  410  -0.1449855E  02 

421  -0. 1723293E  02  422  •0.8886253E  01 

425  -0.8670897E  01  426  -0.8596462E  01 

429  -0.838U05E  01  430  -0.8308662E  01 

433  -0.8091311E  01  434  -0.6018870E  01 

437  -0.7801521E  01  438  0.8289S16E  00 

441  -0. 1143589E  02  442  -0.3019248E  01 
445  -0.2601900E  01  446  -0.2729461E  01 

449  -0.2512121E  01  450  -0.2439681E  01 

453  -0.2222334E  01  454  -0.2149894E  01 

457  -0. 1932574E  01  458  0.6697780E  01 

461  -0.2207393E  02  462  -0.2601007E  02 
465  >0.2554205E  02  466  -0.2536600E  02 
469  -0.2491792E  02  470  -0.2427821E  02 
473  -0.2385416E  02  474  -0.2371281E  02 
477  -0.2360727E  02  478  -0.2358534E  02 

481  -0.1650379E  01  482  -0.1766037E  01 

485  -0.2111933E  01  486  -0.2227591E  01 

489  -0.2574018E  01  490  -0.2689136E  01 

493  -0.3035561E  01  494  -0.3150682E  01 

497  -0.3497104E  01  498  -0.3612759E  01 

501  -0.3858655E  01  502  -0.3940907E  01 

505  -0.4187531E  01  506  -0.4168132E  01 

509  -0.3761486E  01  510  -0.3625919E  01 

513  -0.3960689E  01  514  -0.3962317E  01 

517  -0.3831402E  01  518  -0.3748123E  01 

521  -0.3498280E  01  522  -0.3415006E  01 

525  -0.3165185E  01  526  -0.3081906E  01 

529  -0.2910077E  01  530  -0.2904152E  01 

533  -0.2886378E  01  534  -0.2880454E  01 

537  -0. 2862686 E  01  538  -0.2856759E  01 

541  -0.2838989E  01  542  -0.2833064E  01 

545  -0.2815291E  01  546  -0.2809366E  01 

549  -0.2791595E  01  550  -0.2785671E  01 

553  -0.5993045E  01  554  -0.5968723E  01 

557  -0.5895751E  01  558  -0.5871424E  01 

561  -0.5804512E  01  562  -0.5782413E  01 

565  -0.5716110E  01  566  -0.9731809E  01 

569  -0.8833549E  01  570  -0.8846880E  01 

573  -0. 8989841 E  01  574  -0.9060380E  01 

577  -0.1711795E  02  578  -0.1722049E  02 

581  -0. 1716194E  02  582  -0.1714156E  02 
585  -0. 1708034E  02  586  -0.1705995E  02 

589  -0. 1699873E  02  590  -0.1697832E  02 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

403  -0. 1 113132E  02  404  -0.1111229E  02 
407  -0. 1406404E  02  408  -0.1382692E  02 
411  -0. 1446269E  02  412  -0.14S3500E  02 
415  -0.1483257E  02  416  -0.1464467E  02 
419  -0. 1470986E  02  420  -0.1697525E  02 
423  -0.8815800E  01  424  -0.8743351E  01 
427  -0.8526005E  01  428  -0.6453561E  01 
431  -0.8236202E  01  432  -0.8163757E  01 
435  -0. 7946420E  01  436  -0.7873969E  01 
439  0.9014019E  00  440  -0.2564211E  00 
443  -0.2946792E  01  444  -0.2874346E  01 
447  -0.2657001E  01  448  -0.2584563E  01 
451  -0.2367223E  01  452  -0.2294783E  01 
455  -0.20774S3E  01  456  -0.2005014E  01 
459  0.6770112E  01  460  0.5612514E  01 
463  -0.2585408E  02  464  -0.2569807E  02 
467  -0.2522997E  02  468  -0.2507396E  02 
471  -0.2413689E  02  472  -0.2399551E  02 
475  -0. 23651 08E  02  476  -0.2362917E  02 
479  -0.2356349E  02  480  -0.2354152E  02 
483  -0.1881156E  01  484  -0.1996813E  01 
487  -0.2342706E  01  488  -0.2458366E  01 
491  -0.2804790E  01  492  -0.2919908E  01 
495  -0.3266336E  01  496  -0.3381452E  01 
499  -0.3694283E  01  500  -0.3776469E  01 
503  -0.4023097E  01  504  -0.4105284E  01 
507  -0.4032588E  01  508  -0.389703QE  01 
511  -0.3957409E  01  512  -0.3959044E  01 
515  -0.3963974E  01  516  -0. 39146 77E  01 
519  -0.3664834E  01  520  -0.3581556E  01 
523  -0.3331738E  01  524  -0.3248462E  01 
527  -0.2998629E  01  528  -0.2915336E  01 
531  -0.2898230E  01  532  -0.2892300E  01 
535  -0.2874537E  01  536  -0.2868608E  01 
539  -0. 2850841 E  01  540  -0.2844913E  01 
543  -0.2827139E  01  544  -0.2821212E  01 
547  -0.2803442E  01  548  -0.2797520E  01 
551  -0.6041701E  01  552  -0.6017377E  01 
555  -0.5944400E  01  556  -0.5920074E  01 
559  -0.5848717E  01  560  -0.5826615E  01 
563  -0. 576031 7E  01  564  -0.5738214E  01 
567  -0.8806905E  01  568  -0.8820228E  01 
571  -0.8860196E  01  572  -0.8919337E  01 
575  -0.9130865E  01  576  -0.1701540E  02 
579  -0. 1720274E  02  580  -0.1718234E  02 
583  -0.1712112E  02  584  -0.1710074E  02 
587  -0.1703954E  02  588  -0.1701912E  02 


Figure  16  presents  the  relative  importance  vector 
associated  with  the  algorithm  used  to  calculate  the  detection 
statistics  presented  in  Figure  14  and  Table  3.  This  figure 
plots  each  of  the  385  measurements  listed  in  Table  2  and 
their  relative  importance  for  incipient  engine  failures. 

We  have  also  indicated  by  arrows  and  call  outs  some  of  the 
more  important  contributors  to  this  decision.  Notice, 
that  the  oil  analysis  presents  the  most  significant  informa¬ 
tion  used  for  this  diagnosis  although  the  test  cell  is  also 
very  important  and  some  importance  is  associated  with  some 
of  the  3M  measurements.  Detail  analysis  of  the  important 
wear  metal  concentrations  shows  the  measurements  taken  a 
few  hours  before  the  failure  are  more  important  than  those 
immediately  prior  to  the  failure.  This  is  an  artifact 
which  is  also  due  to  the  interpolation  discussed  above.  Since 
this  artifact  will  not  be  present  in  the  real  world  application 
we  can  also  expect  improvement  in  performance  due  to  this 
effect. 


The  measurement  number  shown  along  the  abscissa  is  the 
same  measurement  number  identified  in  Table  1.  The  magnitude 
is  a  relative  magnitude  add  it  is  the  absolute  value  of  this 
magnitude  which  determines  the  importance.  Thus,  a  measure¬ 
ment  having  a  value  of  minus  2  is  equally  important  to  a 
measurement  having  a  value  of  plus  2.  For  further  discussion 

of  the  relative  vector,  the  reader  is  referred  to  References 
(1),  (2)  and  to  Appendix  A.  To  further  assist  in  identifying 
the  measurements,  the  information  presented  in  Figure  16  is 
repeated  in  Table  4,  where  each  relative  importance 
is  preceded  by  an  interger  identifying  the  measurement. 

This  measurement  identifying  interger  can  be  used  to  relate 
the  measurement  number  (also  shown  as  the  abscissa  on 
Figure  16)  to  the  measurement  name  given  in  Table  1. 
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FIGURE  -  16 

RELATIVE  IMPORTANCE  VECTOR  FOR  INCIPIENT  ENGINE 
FAILURE  DETECTION  ALGORITHM  (590  CASE  STUDY) 


II 


TABLE  4 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  DETECTING  ENGINE  FAILURES 
MEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCE 


1  0.8934487E 

00 

2  0.0 

3 

-0.6498448E 

00 

4 

0.0 

5  0 . 9099075E 

00 

6  -0. 1717615E 

00 

7 

-0.4472 955E 

00 

8 

0.2634745E 

00 

9  -0.1277394E 

01 

10  -0.9848390E 

00 

11 

0. 1086377E 

00 

12 

0.3694587E 

00 

13  0.4732450E 

00 

14  -0.2053033E 

00 

15 

0.7714164E- 

-02 

16 

-0.9216449E 

00 

17  -0. 1347072E 

01 

18  -0. 1388880E 

01 

19 

-0.1373728E 

01 

20 

0. 1 170446E 

01 

21  0.1202013E 

01 

22  0. 1220170E 

01 

23 

0. 1208627E 

01 

24 

0.6374776E 

00 

25  -0.4463237E 

00 

26  0.4577616E 

00 

27 

0. 1093096E 

01 

28 

0. 1 107224E 

01 

29  0.1019790E 

01 

30  0.9018096E 

00 

31 

0.6454996E 

00 

32 

0.3635876E 

00 

33  O.5390641E 

00 

34  0. 7600539E 

00 

35 

0.8749651E 

00 

36 

0.2965026E- 

-01 

37  -0.3269640E 

00 

38  -0.8094662E 

00 

39 

-0. 1786201E 

00 

40 

0. 1069321E 

01 

41  0.2731076E 

00 

42  0.1810486E 

00 

43 

0.2656019E 

00 

44 

0. 1285028E 

01 

45  -0.1489392E 

00 

46  0.4787353E-01 

47 

0.3277532E 

00 

48 

0. 1206516E 

01 

49  -0.7207189E 

00 

50  -0.8740205E 

00 

51 

-0.66 786 39E 

00 

52 

0.3049996E 

00 

53  -0.9890826E 

00 

54  -0.9450681E 

00 

55 

-0. 7808443E 

00 

56 

0. 1 108752E 

01 

57  0.4163638E 

00 

58  0.3261063E 

00 

59 

0.4185230E 

00 

60 

0. 859565 1E- 

-01 

61  -0. 4304591 E 

00 

62  -0. 1812835E 

00 

63 

-0. 1204229E 

00 

64 

0. 1276188E 

01 

65  0. 1S62604E 

00 

66  0.3313220E 

00 

67 

0.6770234E 

00 

68 

-0.2969453E 

00 

69  -0. 1200371E 

01 

70  -0. 1712771E 

01 

71 

-0.1685382E 

01 

72 

0. 2067771 E 

00 

73  -0.I492722E 

01 

74  -0.5618961E 

00 

75 

0.9233391E- 

-01 

76 

0.3375543E 

00 

77  -0.5767193E 

00 

78  -0.4696499E 

00 

79 

-0.3460153E 

00 

80 

-0.3282860E 

00 

81  -0. 1955249E 

00 

82  0. 1595036E 

00 

83 

0. 1886366E 

00 

84 

0. 1345268E 

00 

85  0.5871 36 7E 

00 

86  0.7537163E 

00 

87 

0. 1044842E 

01 

88 

0.5486360E 

00 

89  0.5457790E 

00 

90  0.4651310E 

00 

91 

0.5464313E 

00 

92 

0. 1123170E 

00 

93  -0.5767986E 

00 

94  0.5556448E 

00 

95 

0. 1270509E 

01 

96 

-0.4132727E 

00 

97  0. 1500224E 

00 

98  -0.8528614E-01 

99 

-0. 1073871 E 

00 

100 

-0.2450I96E 

00 

101  -0.3136359E 

00 

102  -0.8042147E 

00 

103 

-0.1770408E 

00 

104 

0.9142579E 

00 

105  0.3247203E 

00 

106  0.3814529E 

00 

107 

0.5206566E 

00 

108 

0.7337209E 

00 

109  0.5267778E 

00 

110  0.5212370E-02 

111 

0.4746042E 

00 

112 

0.6676615E 

00 

113  0.4959108E 

00 

114  0.8788270E-01 

115 

0.6595090E 

00 

116 

-0.9385188E 

00 

117  0.5753794E 

00 

118  0.4473507E 

00 

119 

0 . 3659686E 

00 

120 

-0.8192980E 

00 

121  -0.1611876E 

00 

122  -0..8273473E 

00 

123 

-0.7775759E 

00 

124 

0.9729446E 

00 

125  0.4732988E 

00 

126  0.136U56E 

00 

127 

0.2719671E 

00 

128 

-0. 1532264E 

00 

129  -0.2840513E 

00 

130  -0>1730956E  01 

131 

-0.2909893E- 

-01 

132 

0.8146527E 

00 

133  0. 1345251E 

01 

134  0 . 3652986E 

00 

135 

0.1 159966E 

01 

136 

-0.1934310E 

01 

137  -0.7559242E 

00 

138  0.7424133E  00 

139 

0.1657385E 

01 

140 

0.3500538E 

01 

141  -0.5431552E 

00 

142  -0 . 3004225E-0 1 

143 

-0.1362413E 

01 

144 

0.0 

145  0.0 

146  0.0 

147 

0.7463484E 

00 

148 

0.2084712E 

01 

149  -0.8832578E- 

-02 

150  0.1900508E 

01 

151 

-0.6280541E- 

-01 

152 

0.0 

153  -0.4027243E 

00 

154  -0.147471 IE 

01 

155 

0.7919631E 

00 

156 

0.2710629E 

01 

157  0. 698930 3E 

00 

158  -0.3300804E-01 

159 

-0.4526460E 

00 

160 

0.3991086E 

00 

161  0.3402334E 

00 

162  -0.1602596E 

01 

163 

0 . 7855083E 

00 

164 

-0.6135349E-01 

165  0.3132734E 

00 

166  -0.1110028E 

00 

167 

-0.4718794E 

00 

168 

0. 85291 96E 

00 

169  0.3351340E 

01 

170  0.6099386E 

00 

171 

0.9267356E 

00 

172 

0. 1996185E-01 

173  -0.4402370E 

00 

174  -0.3176594E 

00 

175 

0 . 18368922 

01 

176 

-0.2184756E 

00 

177  0 . 2848958E 

00 

178  0.1058579E 

01 

179 

0.5150238E-01 

180 

0. 1582587E 

01 

181  0.6824994E-01 

182  0.3047652E 

00 

183 

-0.2627738E 

00 

184 

-0. 1147137E 

01 

185  -0.1435264E 

01 

186  0.7971256E 

00 

187 

0 . 1818869E 

01 

188 

-0.2132401E 

01 

189  -0. 1972976E 

01 

190  0.2533190E- 

-01 

191 

0. 9874731 E 

00 

192 

0.8319280E 

00 

193  0. 1224737E 

01 

194  -0. 1796666E 

00 

195 

-0. 1440755E 

01 

196 

0 . 2690501E 

00 

197  0.9372038E- 

-01 

198  -0. 1600819E 

01 

199 

-0. 1218620E 

01 

200 

0.2340994E 

00 
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TABLE  4 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  DETECTING  ENGINE  FAILURES 

CONTINUED 


MEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCEMEAS  REL.  IMPORTANCE 

) 


201 

-0. 337154 9E- 

•01 

202 

-0. 1533018E 

00 

203 

0.9925084E 

00 

204 

0. 1502901E 

01 

205 

0. 1678898E 

01 

206 

0. 1671430E 

01 

207 

0. 1473691E 

01 

208 

0.5771832E 

00 

209 

-0.2173101E 

01 

210 

-0. 1318635E 

01 

211 

-0. 1366706E 

01 

212 

0.2240350E 

01 

213 

0. 101666 7E 

01 

214 

0. 1680344E 

01 

215 

0. 1515994E 

01 

216 

0. 1769063E 

00 

217 

-0 . 3302565E 

00 

218 

0. 1476984E 

01 

219 

0. 1206017E 

00 

220 

0 . 7947659E 

00 

221 

0. 1641100E 

01 

222 

0.9001307E 

00 

223 

0.3271212E 

01 

224 

-0.9850897E 

00 

225 

-0. 1294182E 

01 

226 

0.2048883E 

01 

227 

0.U23338E 

01 

228 

0.9859306E 

00 

229 

-0.4109336E 

00 

230 

-0. 1596781E 

00 

231 

-0.2385519E 

00 

232 

0.4354818E- 

-01 

233 

-0.2577906E 

00 

234 

-0.9575542E 

00 

235 

-0.2692174E 

01 

236 

-0.2107765E 

00 

237 

-0.4705226E 

00 

238 

-0.2111285E 

00 

239 

-0. 1482300E 

01 

240 

-0 .51731 10E 

00 

241 

0.2136491E 

01 

242 

0. 7269270E 

00 

243 

-0.2928626E 

01 

244 

0.7575279E 

00 

245 

0. 1977688E 

01 

246 

-0.9731436E- 

•01 

247 

0. 564984 9E- 

-01 

248 

-0.2019858E 

01 

249 

-0. 1217217E 

01 

250 

0.5090673E 

01 

251 

0.6460260E 

00 

252 

0.8938 382E 

00 

253 

0. 1923118E 

01 

254 

0. 1570183E 

01 

255 

0.9461408E 

00 

256 

-0.5045853E 

00 

257 

-0. 47831 90E 

00 

258 

-0.2016536E 

01 

259 

-0.2231807E 

01 

260 

-0.8600841E 

00 

261 

-0.6929082E 

00 

262 

-0.8039590E 

00 

263 

0. 41084 95E 

00 

264 

-0 . 1664844E 

01 

265 

-0. 1244748E 

01 

266 

0.3525545E 

00 

267 

-0. 1224033E 

01 

268 

-0. 1053982 E 

00 

269 

-0.3338456E 

00 

270 

0.8495033E 

00 

271 

0.2577494E 

01 

272 

0.8577981E 

00 

273 

-0.3863424E 

00 

274 

0.3168389E 

01 

275 

0.3307714E 

00 

276 

0.8285038E 

00 

277 

0 . 9797426E 

00 

278 

-0.2144802E 

00 

279 

0.0 

280 

0.0 

281 

-0.7713985E 

00 

282 

0.0 

283 

-0.5143262E 

00 

284 

-0. 1262039E 

00 

285 

0.0 

286 

0. 1057504E 

01 

287 

0.6124240E 

00 

288 

-0.1 14281 OE 

01 

289 

0.0 

290 

0.0 

291 

0.6648805E 

00 

292 

0.0 

293 

0.0 

294 

0. 1057504E 

01 

295 

0.0 

296 

0.0 

297 

0.1109974E 

01 

298 

0.9752331E 

00 

299 

0.0 

300 

0.0 

301 

0.0 

302 

-0.5143262E 

00 

303 

-0. 1166491E 

01 

304 

0.0 

305 

0.2370443E 

00 

306 

0.0 

307 

0.0 

308 

0.0 

309 

0.0 

310 

0.0 

311 

0.1057504E 

01 

312 

0.6906831E 

00 

313 

-0. 1201338E 

01 

314 

0.5659661E 

00 

315 

0.0 

316 

0.0 

317 

0.6159479E 

00 

318 

0 . 2845502E 

00 

319 

0.0 

320 

0.0 

321 

0.0 

322 

0.0 

323 

0.4649846E 

00 

324 

0.0 

325 

0.2308410E 

00 

326 

0.0 

327 

0.0 

328 

0.0 

329 

0.0 

330 

0.0 

331 

0.0 

332 

0.0 

333 

-0. 1295054E 

01 

334 

0.565966 IE 

00 

335 

0.0 

336 

0.0 

337 

0.6725259E 

00 

338 

0.0 

339 

0. 1085712E 

00 

340 

0.2845502E 

00 

341 

0. 1426241E 

01 

342 

0.0 

343 

0.0 

344 

0.0 

345 

0.0 

346 

0.0 

347 

0.0 

348 

0.0 

349 

0.0 

350 

0.0 

351 

-0.6692418E 

00 

352 

0.0 

353 

-0.1295054E 

01 

354 

0.2845502E 

00 

355 

0. 1063651E 

01 

356 

0.0 

357 

0.0 

358 

0.0 

359 

0.0 

360 

0.0 

361 

0.0 

362 

0.0 

363 

0.0 

364 

0.0 

365 

0. 1844683E 

00 

366 

0.2093776E 

01 

367 

-0. 1349670E 

01 

368 

-0. 1172079E 

01 

369 

0. 1450625E 

01 

370 

0.0 

371 

0.0 

372 

0.0 

373 

0.0 

374 

0.0 

375 

0. 7574350E- 

■01 

376 

0.2014798E 

01 

377 

-0.1 3496 70E 

01 

378 

-0.6577538E 

00 

379 

0.4699914E-01 

380 

0.0 

381 

0.0 

382 

0.0 

383 

0.0 

384 

0.0 

385  0.8931599E  00 
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One  problem  which  may  occur  in  developing  of  the 
incipient  failure  detection  algorithms  presented  in  Figures 
14  and  15  is  that  if  a  failure  has  occurred  and  has  not  been 
observed  or  has  not  been  recorded  in  either  a  3M  or  NOAP 
data  bases,  we  would  not  have  had  it  in  the  training  data. 
Furthermore,  the  decision  as  to  when  an  engine  changes  from 
good  to  bad  prior  to  the  occurrence  of  a  failure  is  rather 
arbitrary  even  when  we  leave  a  reasonable  gap  between  these 
two.  This  is  clear  from  the  fact  that  some  engine  failures 
were  anticipated  by  as  much  as  160  engine  hours  while  others 
only  20  hours  in  advance  of  the  failure. 


An  approach  to  overcome  these  difficulties  is  to  examine 
Figure  14  and  use  it  to  define  the  occurrence  of  the  failures. 
That  is,  if  Figure  14  shows  that  we  have  been  able  to  anticipate 
a  failure  earlier  than  we  had  thought,  this  additional  data 
should  be  put  in  the  failure  class  of  the  training  data  and 
conversely  if  the  failure  could  not  be  detected  until  much 
closer  than  we  had  anticipated  more  of  these  cases  should  be 
put  in  the  good  class.  Furthermore,  if  there  are  indications 
of  failures  which  we  did  not  see,  these  should  be  left  out 
of  the  algorithm  development  since  the  failure  may  have 
occurred  and  not  been  correctly  reported.  That  is,  the  3M 
data  cannot  be  taken  as  100%  reliable!  When  this  is  accomplished 
one  expects  to  significantly  reduce  the  probability  of  error 
associated  with  the  new  algorithm.  We  define  this  procedure 
as  the  bootstrap  procedure. 

This  "bootstrapping"  was  performed  on  the  present  data 
set  and  the  detection  statistic  calculated  from  the  resulting 
algorithm  is  presented  in  Figure  17  and  Table  5.  Figure  17 
and  Table  5  have  the  same  format  as  Figure  14  and  Table  3. 

The  net  result  of  the  bootstrap  was  to  improve  the  probability 
of  error  from  .14  to  .033.  Thus,  the  calls  of  failures  are 
much  stronger  than  they  were  in  Figure  14,  however,  we  have  lost 
the  ability  to  see  some  of  the  failures.  Again,  the  detection 
statistics  listed  in  table  5  are  10  less  than  those  plotted  in 


FIGURE  17  DETECTION  STATISTIC  FOR 
BOOTSTRAP  DETECTION  OF  INCIPIENT  ENGINE  FAILURES 


TABLE  5 

DETECTION  STATISTIC  FOR  BOOT  STRAP  INCIPIENT  ENGINE  FAILURE  DETECTOR 
HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


1  -0. 1962062E  02 
5  -0 . 1949530E  02 
9  -0.1934508E  02 
13  -0.1915460E  02 
17  -0. 1459046E  02 
21  -0.1717016E  02 
25  -0.1647765E  02 
29  -0.1918619E  02 
33  -0. 1763937E  02 
37  -0. 1669809E  02 
41  -0. 1554239E  02 
45  -0.1 3736 75E  02 
49  -0. 1244738E  02 
53  -0. 1Z41985E  02 
57  -0. 1249442E  02 
61  -0. 1267003E  02 
65  -0. 1261 107E  02 
69  -0.1251757E  02 
73  -0.2023820E  01 
77  -0. 1706642E  01 
81  -0.2149347E  01 
85  -0. 1683126E  01 
89  -0.2444041E  01 
93  -0. 1132482E  01 
97  0.3622551E  00 

101  0.2041356E  01 

105  0. 15031 95E  01 

109  -0.5658771E  01 
113  -0.6095848E  01 
117  -0.7821368E  01 
121  -0.8913341E  01 
125  -0.9632792E  01 
129  -0.9926025E  01 
J33  -0.9453003E  01 
137  -0.8979988E  01 
141  -0.8506972E  01 
145  -0.8408436E  01 
149  -0.2523170E  02 
153  -0.2486789E  02 
157  -0.2201480E  02 
161  -0.2202075E  02 
165  -0.2061734E  02 
169  -0.2076224E  02 
173  -0. 1952287E  02 
177  -0. 1971036E  02 
181  -0.2113077E  02 
185  -0.2050146E  02 
189  -0.2039561E  02 
193  -0.2039073E  02 
197  -0.2038586E  02 


2  -0. 1959218E  02 
6  -0. 1946060E  02 
10  -0. 1929747E  02 
14  -0. 1910696E  02 
18  -0. 1409260E  02 
22  -0. 1657716E  02 
26  -0. 1901791E  02 
30  -0. 1873013E  02 
34  -0. 1678902E  02 
38  -0. 1554325E  02 
42  -0. 1502516E  02 
46  -0. 1302389E  02 
50  -0. 1275542E  02 
54  -0. 1225389E  02 
58  -0. 1260929E  02 
62  -0. 1265682E  02 
66  -0. 1259268E  02 
70  -0.1248887E  02 
74  -0.1944521E  01 
78  -0.1627329E  01 
82  -0.2 146 74 7E  01 
86  -0. 1732603E  01 
90  -0.7339540E  00 
94  -0.1120761E  01 
98  -0. 1080265E  00 
102  0.1507122E  01 

106  -0.2752731E  01 
110  -0.4679473E  01 
114  -0.7002342E  01 
118  -0.8106318E  01 
122  -0.9182364E  01 
126  -0.9735392E  01 
130  -0.9807775E  01 
134  -0.933476IE  01 
138  -0.8861744E  01 
142  -0.8388722E  81 
146  -0. 1032615E  02 
150  -0.2522549E  02 
154  -0.2385139E  02 
158  -0.2200291E  02 
162  -0.2083493E  02 
166  -0.2063036E  02 
170  -0. 1966783E  02 
174  -0. 1956975E  02 
178  -0. 1975725E  02 
182  -0.2080798E  02 
186  -0.2039926E  02 
190  -0.2039439E  02 
194  -0.2038951E  02 
198  -0.2038463E  02 


3  -0 . 195631 OE  02 
7  -0. 1942331E  02 
11  -0. 1924983E  02 
15  -0. 1905936E  02 
19  -0. 1436052E  02 
23  -0. 1778517E  02 
27  -0. 1935194E  02 
31  -0.1830605E  02 
35  -0. 1694510E  02 
39  -0.1576331E  02 
43  -0.1421644E  02 
47  -0. 1225700E  02 
51  -0. 1275175E  02 
55  -0. 1223000E  02 
59  -0. 1252314E  02 
63  -0.1264361E  02 
67  -0. 1257429E  02 
71  -0.9840500E  01 
75  -0.1865225E  01 
79  -0. 1677001E  01 
83  -0.2034892E  01 
87  -0.S975485E  00 
91  0.4106972E  00 

95  -0.7811863E  00 
99  0.3842993E  00 
103  0.6000788E  00 
107  -0.3791129E  01 
111  -0.4092389E  01 
115  -0 . 7908897E  01 
119  -0.8375325E  01 
123  -0.9427608E  01 
127  -0.9837989E  01 
131  -0.9689524E  01 
135  -0.9216508E  01 
139  -0.8743466E  01 
143  -0.8270462E  01 
147  -0.2524408E  02 
151  -0.2521861E  02 
155  -0.2223325E  02 
159  -0.2199867E  02 
163  -0.2078619E  02 
167  -0.2064336E  02 
171  -0 . 1697385E  02 
175  -0.1961662E  02 
179  -0.19804UE  02 
183  -0.2070581E  02 
187  -0.2039804E  02 
191  -0.2039317E  02 
195  -0.2038829E  02 
199  -0.2038342E  02 


4  -0. 1952919E  02 
8  -0. J938602E  02 
12  -0. 1920221E  02 
16  -0. 1490845E  02 
20  -0. 1791605E  02 
24  -0. 1717883E  02 
28  -0. 1948158E  02 
32  -0. 1792888E  02 
36  -0.1711482E  02 
40  -0. 1588961E  02 
44  -0. 1385991E  02 
48  -0 . 1213933E  02 
52  -0.1258580E  02 
56  -0. 1237954E  02 
60  -0. 1246335E  02 
64  -0. 1262946E  02 
68  -0. 1254627E  02 
72  -0.2103108E  01 
76  -0. 1785935E  01 
80  -0. 1726657E  01 
84  -0. 1876164E  01 
88  -0. 1309B23E  01 
92  -0. 1856200E  00 
96  -0.1067675E  01 
100  0. 1414097E  01 

104  0.9097822E  00 

108  -0.4829443E  01 
112  -0.5189363E  01 
116  -0.8381500E  01 
120  -6.6444333E  01 
124  -0.9S30196E  01 
128  -0 . 9944366E  01 
132  -0.9571272E  01 
136  -0.9098241E  01 
140  -0.8625222E  01 
144  -0.8424775E  01 
148  -0.2523788E  02 
152  -0.2521143E  02 
156  -0.2202670E  02 
160  -0.2200970E  02 
164  -0.2060432E  02 
168  -0.2070280E  02 
172  -0. 1947600E  02 
176  -0. 1966348E  02 
180  -0. 1985098E  02 
184  -0.2060365E  02 
188  -0.2039682E  02 
192  -0.2039195E  02 
196  -0.2038708E  02 
200  -0.2038219E  02 
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TABLE  5 

DETECTION  STATISTIC  FOR  BOOT  STRAP  INCIPIENT  ENGINE  FAILURE  DETECTOR 

CONTINUED 

HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


201  -0.2038100E  02 
205  -0.2037611E  02 
209  -0.2037123E  02 
213  -0.2036635E  02 
217  -0.2226163E  02 
221  -0.2224698E  02 
225  -0.2223236E  02 
229  -0. 1283721E  02 
233  -0.1065165E  02 
237  -0.8466069E  01 
241  -0.8599792E  01 
245  -0.8593920E  01 
249  -0.9007354E  01 
253  -0.2368979E  02 
257  -0.2327542E  02 
261  -0.2464409E  02 
265  -0.3130577E  02 
269  -0.3007007E  02 
273  0.444 3691 E  01 
277  0.3718952E  01 
281  0.3488029E  01 
285  0. 34540 18E  01 
289  0.4017258E  01 
293  0.2260973E  01 
297  0.8799934E  00 
301  0. 1469557E  01 
305  0. 1303401E  01 
309  0.6901312E  00 
313  -Q.2675012E  02 
317  -0.3069557E  02 
321  -0.3039143E  02 
325  -0.3019638E  02 
329  -0.3018452E  02 
333  0.2015784E  01 
337  0. 1568163E  01 
341  0.1198826E  01 
345  0. 1142977E  01 
349  0. 1143241E  01 
353  0. 1143525E  01 
357  0. 1226315E  01 
361  -0. 1362100E  02 
365  -0.1381141E  02 
369  -0.1407580E  02 
373  -0. 1434023E  02 
377  -0. 1457657E  02 
381  -0. 1959760E  02 
385  -0.182296QE  02 
389  -0. 1628362E  02 
393  -0. 1433764E  02 
397  -0.1239165E  02 


202  -0.2037976E  02 
206  -0.2037488E  02 
210  -0.2037C01E  02 
214  -0.2036511E  02 
218  -0.2225798E  02 
222  -0.2224333E  02 
226  -0. 1431299E  02 
230  -0.1229082E  02 
234  -0.1010525E  02 
238  -0. 7919685E  01 
242  -0.8598331E  01 
246  -0.8592455E  01 
250  -0.9145660E  01 
254  -0.2358624E  02 
258  -0.2317183E  02 
262  -0.3103990E  02 
266  -0.3095833E  02 
270  -0.3009959E  02 
274  0.520B057E  01 
278  0.3661224E  01 
282  0.3430308E  01 
286  0.35 94831 E  01 
290  0.3945745E  01 
294  0. 1937800E  01 
298  0.8390166E  00 
302  0.2024706E  01 
306  0. 1259049E  01 
310  0.6457253E  00 
314  -0.2667451E  02 
318  -0.3061824E  02 
322  -0.3031581E  02 
326  -0.3019342E  02 
330  -0.3016153E  02 
334  0. 1903860E  01 
338  0.1456232E  01 
342  0.1176446E  01 
346  0. 1143036E  01 
350  0.1143316E  01 
354  0.U43597E  01 
358  0. 1230757E  01 
362  -0. 1366862E  02 
366  -0. 1387751E  02 
370  -0. 1414192E  02 
374  -0. 1440637E  02 
378  -0. 1454457E  02 
382  -0. 1948567E  02 
386  -0. 1774310E  02 
390  -0. 1579714E  02 
394  -0. 1385114E  02 
398  -0. 1235971E  02 


203  -0 . 2037854E  02 
207  -0.2037367E  02 
211  -0.2036880E  02 
215  -0.203O391E  02 
219  -0.2225432E  02 
223  -0.2223967E  02 
227  -0. 1393000E  02 
231  -0. 1 174444E  02 
235  -0 . 955886 IE  01 
239  -0. 7373298E  01 
243  -0.8596866E  01 
247  -0.8730735E  01 
251  -0.2338123E  02 
255  -0.2348260E  02 
259  -0.2306827E  02 
263  -0.3118915E  02 
267  -0.3001 106E  02 
271  0.2914948E  01 
275  0.5972332E  01 
279  0.3603477E  01 
283  0.3372602E  01 
287  0.3735624E  01 
291  0.4288360E  01 
295  0. 1614618E  01 
299  0. 1712259E  01 
303  0.2182888E  01 
307  0.1214696E  01 
311  -0.2690131E  02 
315  -0.2659894E  02 
319  -0.3054265E  02 
323  -0.3024020E  02 
327  -0.3019044E  02 
331  0.2401726E  01 
335  0. 1791956E  01 
339  0. 1344314E  01 
343  0.1 154060E  01 
347  0 . 1 143112E  01 
351  0 . 1I43385E  01 
355  0 . 1 176476E  01 
359  0 . 1235199E  01 
363  -0. 1371619E  02 
367  -0. 1394363E  02 
371  -0.1420S03E  02 
375  -0. 1447251E  02 
379  -0. 1451252E  02 
383  -0. 191 90 1 1 E  02 
387  -0. 1725661E  02 
391  -0. 1531064E  02 
395  -0. 1336464E  02 
399  -0. I187321E  02 


204  -0.2037733E  02 
208  -0.2037245E  02 
212  -0.2036755E  02 
216  -0 . 2226527E  02 
220  -0.2225067E  02 
224  -0.2223601E  02 
228  -0 . 1338360E  02 
232  -0.1P9805E  02 
236  -0.9012458E  01 
240  -0.6826912E  01 
244  -0.6595394E  01 
248  -0.8869055E  01 
252  -0.2353548E  02 
256  -0.2337906E  02 
260  -0 . 2457755E  02 
264  -0.3133838E  02 
268  -0.3004057E  02 
272  0. 367931 7E  01 
276  0.6736633E  01 
280  0 . 3545755E  01 
284  0.3314880E  01 
288  0.3876434E  01 
292  0.2587406E  01 
296  0. 1291432E  01 
300  0.2463924E  01 
304  0. 1743152E  01 
308  0. 1170332E  01 
312  -0.2682573E  02 
316  -0.3205127E  02 
320  -0.3046701E  02 
324  -0. 301646 3E  02 
328  -0.3018748E  02 
332  0.2195199E  01 
336  0. 1660039E  01 
340  0. 1232411E  01 
344  0. 1142900E  01 
348  0.1 143172E  01 
352  0 . 1 143455E  01 
356  0. 1221868E  01 
360  0. 1239654E  01 
364  -0.1 376377E  02 
368  -0. 1400972E  02 
372  -0. 1427411E  02 
376  -0. 1453877E  02 
380  -0. 1448045E  02 
384  -0. 1871608E  02 
388  -0 . 1677010E  02 
392  -0. 1482414E  02 
396  -0.1287815E  02 
400  -0. 1263375E  02 
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TABLE  5 

DETECTION  STATISTIC  FOR  BOOT  STRAP  INCIPIENT  ENGINE  FAILURE  DETECTOR 

CONTINUED 

MOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


401  -0. 1263817E  02 
405  -0. 1474804E  02 
409  -0.1467718E  02 
413  -0. 1524905E  02 
417  -0.1450413E  02 
421  -0.2304174E  01 
425  -0.3584309E  00 
429  -0. 1026778E  00 
433  0. 1530784E  00 
437  0.4088349E  00 
441  0.6920906E  01 
445  0.8944189E  01 
449  0. 9199945E  01 
453  0.9455697E  01 
457  0.9711452E  01 
461  -0. 1796280E  02 
465  -0.1599236E  02 
469  -0. 1478054E  02 
473  -0.135161 IE  02 
477  -0.1342338E  02 
481  -0.1155621E  02 
485  -0.1165663E  02 
489  -0. 1175716E  02 
493  -0.1185758E  02 
497  -0.1195800E  02 
501  -0. 1191818E  02 
505  -0. 1183338E  02 
509  -0.1185334E  02 
513  -0.1121120E  02 
517  -0. 1009565E  02 
521  -0.1016127E  02 
525  -0.1022687E  02 
529  -0.1027780E  02 
533  -0. 1027901E  02 
537  -4.1Q28023E  02 
541  -0.1028145E  02 
545  -0.1028267E  02 
549  -0. 1028389E  02 
553  -0.4019934E  01 
557  -0.3972884E  01 
561  -0.394371 IE  01 
565  -0.3922860E  01 
569  -0.1U3548E  02 
573  -0.1114126E  02 
577  -0.2077744E  02 
581  -0.2085532E  02 
585  -0.2084552E  02 
589  -0.2083571E  02 


402  -0. 1264258E  02 
406  -0. 1475246E  02 
410  -0 . 1490915E  02 
414  -0. 1531308E  02 
418  -0. 15S6096E  02 
422  -0.5502502E  00 
426  -0.2944952E  00 
430  -0. 38 7380 IE -01 
434  0.2170172E  00 
438  0.1820523E-01 
442  0.8752373E  01 
446  0.9008128E  01 
450  0.9263886E  01 
454  0.9519634E  01 
458  0.9320731E  01 
462  -0. 16901 12E  02 
466  -0.1568939E  02 
470  -0.1371130E  02 
474  -0.1345104E  02 
478  -0.1342115E  02 
482  -0.1158137E  02 
486  -0.1 1681 79E  02 
490  -0.1 17822 IE  02 
494  -0.1188262E  02 
498  -0.1198316E  02 
502  -0.U89697E  02 
506  -0. 1178279E  02 
510  -0 . 1 1B7685E  02 
514  -0. 1073548E  02 
518  -0. 1011205E  02 
522  -0.1017767E  02 
526  -0.1024327E  02 
530  -0. 1027810E  02 
534  -0. 1027932E  02 
538  -0. 1026053E  02 
542  -0. 1028175E  02 
546  -0. 1028297E  02 
550  -0. 102841 9E  02 
554  -0.4008173E  01 
558  -0.3961120E  01 
562  -0.3938496E  01 
566  -0. 1096580E  02 
570  -0. 11 14697E  02 
574  -0. 1113054E  02 
578  -0.2085701E  02 
582  -0.2085286E  02 
586  -0.2084306E  02 
590  -0.2083325E  02 


403  -0 . 1264700E  02 
407  -0. 1481272E  02 
411  -0. 1514112E  02 
415  -0 . 1535404E  02 
419  -0. 1632675E  02 
423  -0.4863093E  00 
427  -0.2305545E  00 
431  0. 2520019E-01 
435  0.280954 IE  00 
439  0.8214551E-01 
443  0.8816313E  01 
447  0 . 9072067E  01 
451  0 . 9327823E  01 
455  0 . 9583572E  01 
459  0.9364558E  01 
463  -0. 1659821E  02 
467  -0. 1538645E  02 
471  -0 . 1364623E  02 
475  -0. 1342785E  02 
479  -0.1341893E  02 
483  -0. 1160642E  02 
487  -0.1170684E  02 
491  -0.1180737E  02 
495  -0. 1190779E  02 
499  -0 . 1 196056E  02 
503  -0.118757BE  02 
507  -0.1180631E  02 
511  -0.1216265E  02 
515  -0.1025976E  02 
519  -0.1012846E  02 
523  -0. 1019407E  02 
527  -0.1025967E  02 
531  -0. 1027841E  02 
535  -0. 1027962E  02 
539  -0. 1028084E  02 
543  -0. 1028206E  02 
547  -0.1028327E  02 
551  -0.4043463E  01 
555  -0. 3996408E  01 
559  -0.3954135E  01 
563  -0.3933268E  01 
567  -0. 1111254E  02 
571  -0. 1115842E  02 
575  -0. 1111 971 E  02 
579  -0.2086021E  02 
583  -0.2085040E  02 
587  -0.2084061E  02 


404  -0. 1265143E  02 
408  -0.1464708E  02 
412  -0. 1518504E  02 
416  -0. 1500125E  02 
420  -0. 1663373E  02 
424  -0.4223724E  00 
428  -0. 1666139E  00 
432  0.8914042E-01 
436  0. 3448936E  00 
440  -0. 1232711E  01 
444  0.8880250E  01 
448  0.9136009E  01 
452  0. 9391761E  01 
456  0.9647512E  01 
460  0.8069884E  01 
464  -0. 1629526E  02 
468  -0. 1508352E  02 
472  -0. 13581 18E  02 
476  -0. 134256 3E  02 
480  -0.1341667E  02 
484  -0.U63158E  02 
488  -0.1173200E  02 
492  -0.11B3242E  02 
496  -0.1193283E  02 
500  -0.1193937E  02 
504  -0.1185458E  02 
508  -0. 1182953E  02 
512  -0.1 1686 93E  02 
516  -0. 1007924E  02 
520  -0. 1014487E  02 
"24  -0. 1021047E  02 
528  -0. 102760BE  02 
532  -0. 1027871E  02 
536  -0. 1027993E  02 
540  -0. 1028115E  02 
544  -0.1028236E  02 
548  -0. 1028358E  02 
552  -0.4031700E  01 
556  -0.3984646E  01 
560  -0.3948922E  01 
564  -0.3928076E  01 
568  -0.1112401E  02 
572  -0.1115207E  02 
576  -0.2069785E  02 
580  -0.2085777E  02 
584  -0.2084796E  02 
588  -0.2083815E  02 
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Figure  17.  Notice,  that  G5  has  fallen  below  the  threshold 
but  still  has  a  similar  shape  but  G4  is  now  detected.  It 
also  has  negative  slopes  and  represents  measurements  immediately 
after  the  test  cell.  Thus,  the  discussion  of  G5  is  also 
applicable  in  this  case.  The  reader  is  also  encouraged  to 
“play  with"  adjusting  the  threshold  on  Figures  14  and  17. 

Figure  18  and  Table  6  present  the  relative  importance 
vectors  corresponding  to  Figure  17.  We  see  that  in  the  boot¬ 
strap  case,  the  NOAP  data  is  still  the  most  important  data, 
however,  the  importance  of  the  test  cell  relative  to  the  NOAP 
data  has  increased. 


ADAPT  Scatter  Plot  As  Health  Diagnostic  Tool 

Figure  13  showed  that  the  dominant  groupings  which 
occurred  in  the  plot  of  the  first  two  eigendirections  were 
due  to  the  variation  from  engine  to  engine.  However,  there 
is  also  considerable  variation  within  a  given  engine  when 
the  data  was  obtained  over  relatively  long  time  periods. 

The  two  engines  for  which  this  was  true  have  each  been  en¬ 
closed  in  solid  curves  and  identified  on  Figure  13.  These 
two  engines  are  Serial  #  687021  and  687279.  For  these  two 
engines,  information  is  available  both  when  the  engine  was 
operating  satisfactorily  and  where  an  incipient  failure  was 
anticipated,  we  see  that  there  is  considerable  variation 
especially  in  the  second  eigendirection.  Thus,  if  one  were 
to  replot  the  data  presented  in  Figure  13  for  one  of  these 
engines  on  a  scale  appropriate  to  that  engine  by  itself, 
one  nu^nt  expect  that  more  information  on  the  engine  health 
would  be  revealed.  Figure  19  is  such  a  projection. 

On  Figure  19,  the  symbols  indicate  the  time  from  overhaul. 
That  is.  Symbol  1  occurred  immediately  after  the  overhaul 
followed  by  Symbol  2,  and  so-on  up  through  9.  After  9,  the 
symbols  begin  with  the  letters  thus  the  letter  A  occurs 
immediately  after  9  and  X  occurs  very  near  to  the  time  when 
the  engine  re-enters  the  test  cell  for  another  overhaul. 

There  are  two  of  each  of  the  symbols  on  this  figure.  Using 
these  symbol  identifications,  we  can  trace  the  life  of  the 
engine  on  this  figure.  When  an  incipient  failure  is  likely 
it  tends  to  be  located  in  the  upper  left  hand  portion  of  the 
figure  and  when  the  engine  is  healthy  it  tends  to  be  in  the 
lower  right  hand  corner  of  the  figure. 

Figure  19  is  a  good  illustration  of  the  use  of  the  scatter 
plot  to  identify  incipient  failures.  Th.^s  may  be  considered 
"learning  without  a  teacher"  because  the  general  procedure 
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FIGURE  -18  RELATIVE  IMPORTANCE  VECTOR  FOR 
BOOT  STRAP  INCIPIENT  ENGINE  FAILURE  DETECTOR 


II 


TABLE  6 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  OETECTING  ENGINE  FAILURES  tREVISED) 


MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


1 

0.2350556E 

01 

2  0.0 

3 

-0 . 1560678E 

01 

4 

0.0 

5 

0.4871071E 

00 

6  -0.5647349E 

00 

7 

-0 . 2088690E 

00 

8 

0.2664590E 

00 

9 

-0.2460429E 

01 

10  -0 . 1619416E 

01 

It 

0.7671485E 

00 

12 

0.9370001E 

00 

13 

0 . 1013504E 

01 

14  -0. 1963878E 

00 

15 

-0.1 146592E 

01 

16 

-0. 1035009E 

01 

17 

-0 . 1396945E 

01 

16  -0. 1453564E 

01 

19 

-0. 1382540E 

01 

20 

0.6871 162E 

00 

21 

0.6846229E 

00 

22  0.6926244E 

00 

23 

0.6681978E 

00 

24 

-0.7922701E 

00 

25 

0.7944936E 

00 

26  0.7713941E 

00 

27 

0 . 1249090E 

01 

26 

0.7856145E 

00 

29 

0.2102443E 

01 

30  0. 962862 3E 

00 

31 

0 . 1 133018E 

01 

32 

-0.2564996E 

00 

33 

0. 1626425E 

00 

34  0.34931UE 

00 

35 

0.4499439E 

00 

36 

-0. 1033S66E 

00 

37 

0.6477765E 

00 

38  -0.3474251E 

00 

39 

0.5119747E 

00 

40 

0.6666123E 

00 

41 

0.5946092E 

00 

42  0 . 3572509E 

00 

43 

0.5290286E 

00 

44 

0. 1320721E 

01 

45 

0.2170734E 

00 

46  0.3523064E 

00 

47 

0 . 7052336E 

00 

46 

0 . 1105405E 

01 

49 

-0.2142078E 

00 

50  -0 . 3944266E 

00 

51 

-0.237692 IE 

00 

52 

0 . 1761693E 

00 

53 

-0 . 7326026E 

00 

54  -0.2074368E 

00 

55 

0.2413153E- 

-01 

56 

0.9147494E 

00 

57 

0.7607517E 

00 

58  0.5166691E 

00 

59 

0.6996319E 

00 

60 

0.3716767E 

00 

61 

-0.6361935E 

00 

62  -0.2166954E 

00 

63 

-0.2796462E 

00 

64 

0. 1316726E 

01 

65 

0.5862291E 

00 

66  0.6766234E 

00 

67 

0. 1100093E 

01 

66 

-0.5592723E 

00 

69 

-0.5926327E 

00 

70  -0. 1190993E 

01 

71 

-0.1 33066 7E 

01 

72 

-0. 1 156214E 

01 

73 

-0.1676652E 

01 

74  -0.3390880E 

00 

75 

-0.1292088E 

00 

76 

0.2166092E  «0 

77 

-0. 1075145E 

00 

76  0.4825306E 

00 

79 

0.7514060E 

00 

60 

-0.6071343E 

00 

01 

-0.37690946 

00 

62  -0.S122168E 

00 

83 

-0.5859075E 

00 

64 

0.2730539E 

00 

65 

O.9315013E 

00 

86  0.5169866E 

00 

87 

0.7318604E 

00 

66 

0.5133194E 

00 

69 

0.055936 96 

00 

90  0.4449697E 

00 

91 

0.5902824E 

00 

92 

-0.1172622E 

01 

93 

0.5481842E 

00 

94  -0.1698582E 

00 

95 

0.1605760E 

00 

96 

-0.2119389E 

00 

97 

0.13945116 

01 

98  -0.1475707E 

00 

99 

0.4145646E- 

-01 

100 

-0.2522351E 

00 

101 

0.6616216E 

00 

102  -0.35447106 

00 

103 

0.5059418E 

00 

104 

-0.4377366E 

00 

105 

-0.69467576 

00 

106  -0.6486169E 

00 

107 

-0.5612956E 

00 

106 

0. 1360219E 

01 

109 

0. 1323287E 

01 

110  0.6910790E 

00 

111 

0 . 1445069E 

01 

112 

0.7967729E 

00 

113 

0.7436133E 

00 

114  0. 1502762E-01 

115 

0.8124371E 

00 

116 

-0.7681659E 

00 

117 

0.9231129E 

00 

118  0.9361697E 

00 

119 

0.8572121E 

00 

120 

-0.5083637E 

00 

121 

0 . 3536653E 

00 

122  -0.6691023E 

00 

123 

-0.7653536E 

00 

124 

0.9263667E 

00 

125 

0.6506393E 

00 

126  0.2694795E 

00 

127 

0.2612656E 

00 

126 

0.3944593E- 

•03 

129 

-0. 3764605E 

00 

130  0.5405067E 

00 

131 

0.2434466E 

00 

132 

-0.3409026E- 

-01 

133 

0. 1797749E- 

-01 

134  -0. 1274137E 

00 

135 

0. 1594490E 

00 

136 

-0.2314404E 

01 

137 

-0.3291377E 

00 

138  0. 23084 74E-01 

139 

0. 3993391 E 

00 

140 

0.6236750E 

00 

141 

0. 93944 75E 

00 

142  0. 171324 3E 

01 

143 

0.9994709E- 

-01 

144 

0.0 

145 

0.0 

146  0.0 

147 

0.6714134E 

00 

146 

0.1 1206 76E 

00 

149 

0.3172456E 

00 

150  -0.2164214E 

00 

151 

-0.6112099E- 

-02 

152 

0.0 

153 

-0.6129599E 

00 

154  0.5219650E 

00 

155 

0.7904097E 

00 

156 

0.4523060E 

00 

157 

0.28603S6E 

00 

156  -0.332 76 95E 

00 

159 

-0.2579585E- 

-01 

160 

-0.373623SE 

00 

161 

0. 7746694E 

00 

162  -0. 1309187E 

00 

163 

0.2591209E 

00 

164 

-0.3315979E 

00 

165 

-0.4560606E 

00 

166  -0.6993664E 

00 

147 

-0.7054967E 

00 

166 

0.726641 IE 

00 

169 

0.3305656E 

01 

170  0.6616296E 

00 

171 

0. 1200941E 

01 

172 

0.9194040E- 

-01 

173 

0.4641278E- 

-01 

174  -0.9997463E- 

-01 

175 

0. 35971 23E 

00 

176 

0.2273671E- 

-02 

177 

0. 352496 3E 

00 

176  0. 2203491E 

01 

179 

0.6694516E 

00 

160 

0 . 1 188102E 

01 

181 

0.10734716 

01 

102  0. 1049574E 

01 

163 

0.5502053E 

00 

164 

-0.6468496E 

00 

165 

-0.15755376 

01 

166  0.6066493E-01 

187 

-0.2026812E-01 

166 

-0.6690657E 

00 

169 

-0.1014205E 

01 

190  -0.3542175E 

00 

191 

0. 1063432E 

01 

192 

0.9015971E 

00 

193 

0. 1485291 E 

01 

194  0.6338568E 

00 

195 

0. 1924150E 

00 

196 

0 . 1408242E 

01 

197 

0.1 185410E 

01 

198  0. 1327655E 

01 

199 

0.6454791E 

00 

200 

0.6506033E 

00 

II 

II 
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TABLE  6 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  DETECTING  ENGINE  FAILURES  (REVISED) 

CONTINUED 

MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


201  -0.233696  IE 

00 

202 

0.6099626E 

00 

203 

0 

777804 9E 

00 

204 

0. 1470064E 

00 

205  0. 99996 75E-01 

206 

-0 . 3266416E 

00 

207 

0 

1601763E 

00 

208 

0.5164116E 

00 

209  -0.2952331E 

01 

210 

-0.5667363E 

00 

211 

-0 

1099499E 

01 

212 

0.2189371E 

01 

213  0. 1785126E 

01 

214 

0. 1397624E 

01 

215 

0 

68732 95E 

00 

216 

0.5181375E 

00 

217  -0.1722227E 

00 

218 

0. 1155660E 

01 

219 

0 

7371 26 7E 

00 

220 

0. 38242 14E 

00 

221  0.2834910E 

00 

222 

-0 . 1596912E 

00 

223 

0 

4337677E 

00 

224 

-0.1092763E 

01 

225  -0. 723166 3E 

00 

226 

0. 1076423E 

01 

227 

0 

1361969E 

01 

226 

-0.6132606E-01 

229  -0.2014700E 

00 

230 

-0. 1672661E 

00 

231 

0 

8066 347E 

00 

232 

0.8870563E 

00 

233  0.5299921E 

00 

234 

0.U55257E 

01 

235 

-0 

4862702E 

00 

236 

-0.4921629E 

00 

237  -0.7584127E 

00 

238 

-0.6367360E 

00 

239 

-0 

3103380E 

00 

240 

-0.3330597E 

00 

241  0.2167778E 

01 

242 

0.4943612E 

00 

243 

3586 167E 

00 

244 

0. 1499721E 

01 

245  0. 149454 7E 

01 

246 

0.1459604E 

01 

247 

0 

1021918E 

01 

246 

-0. 1504533E 

01 

249  -0. 1222542E 

01 

250 

0.9934161E 

00 

251 

0 

1001630E 

01 

252 

-0.7121636E 

00 

253  0.4694405E- 

-01 

254 

-0.2005947E 

00 

255 

0 

7430S36E 

00 

256 

-0.2436665E 

00 

257  -0.2101762E 

00 

258 

-0.1836745E 

00 

259 

1722492E 

00 

260 

0. 62330 74E- 

-02 

261  0.5586067E-01 

262 

0.5564931E-OI 

263 

0 

1928283E 

00 

264 

-0.241 141 7E 

01 

265  -0. 1816116E 

01 

266 

-0.3046605E-01 

267 

-0 

1634645E 

01 

268 

-0.6481668E 

00 

269  -0.7212032E 

00 

270 

-0.1367660E 

01 

271 

0 

69871B4E- 

-01 

272 

0. 1715071E 

01 

273  0.6326209E 

00 

274 

0. 2 938306 E 

00 

275 

3666108E- 

-01 

276 

0.5419933E 

00 

277  -0.5450608E-01 

278 

0. 1692515E 

00 

279 

0 

0 

280 

0.* 

281  -0. 14594 76E 

01 

282 

0.0 

283 

0 

2111334E 

00 

264 

-0 . 131bi56E 

00 

285  0.0 

286 

0. 29736 36E 

00 

287 

-0 

4B23576E 

00 

288 

-0.4706759E 

on 

289  0.0 

290 

0.0 

291 

4691060E 

00 

292 

0.0 

293  0.0 

294 

0.2973636E 

oo 

295 

0 

0 

296 

0.0 

297  -0. 1629257E 

00 

298 

0. 1617276E 

01 

299 

0 

0 

300 

0.0 

301  0.0 

302 

0.2U1334E 

00 

303 

70071 08E 

00 

304 

0.0 

305  -0.3324370E 

00 

306 

0.0 

307 

0 

0 

306 

0.0 

309  0.0 

310 

0.0 

311 

0 

29736 36E 

00 

312 

0. 1381245E 

01 

313  -0.I947230E 

01 

314 

0.2338 75 3E 

00 

315 

0 

0 

316 

0.0 

317  -0.8985617E 

00 

318 

0 . 2360320E 

00 

319 

0 

0 

320 

0.0 

321  0.0 

322 

0.0 

323 

-0 

3865244E 

00 

324 

0.0 

325  0.8949564E- 

-01 

326 

0.0 

327 

0 

0 

328 

0.0 

329  0.0 

330 

0.0 

331 

0 

0 

332 

0.0 

333  -0.5439314E 

00 

334 

0.2338753E 

00 

335 

0 

0 

336 

0.0 

337  -0.492631 IE 

00 

338 

0.0 

339 

1386636E 

01 

340 

0.2360320E 

00 

341  0.7106846E 

00 

342 

0.0 

343 

0 

0 

344 

0.0 

345  0.0 

346 

0.0 

347 

0 

0 

346 

0.0 

349  0.0 

350 

0.0 

351 

-0 

72 34656 E 

00 

352 

0.0 

353  -0.5439314E 

00 

354 

0.2360320E 

00 

355 

0 

3574 129E 

00 

356 

0.0 

357  0.0 

358 

0.0 

359 

0 

0 

360 

0.0 

361  0.0 

362 

0.0 

363 

0 

0 

364 

0.0 

365  -0.9999349E 

00 

366 

-0.3870960E 

00 

367 

5509857E 

00 

366 

0.6590769E 

00 

369  -0.8532407E 

00 

370 

0.0 

371 

0 

0 

372 

0.0 

373  0.0 

374 

0.0 

375 

37590 96E 

00 

376 

0.2006377E 

00 

377  -0.5509657E 

00 

378 

0.44 794 34E 

00 

379 

1053S95E- 

-01 

360 

0.0 

381  0.0 

362 

0.0 

383 

0 

0 

304 

0.0 

385  -0.7741756E 

00 

366 

-0.1955916E 

02 

387 

2015359E 

02 

366 

-0.2074605E 

02 

389  -0.2134247E 

02 

390 

-0.2193690E 

02 

391 

22531 34E 

02 

392 

-0.2312576E 

02 

393  -0.2372021E 

02 

394 

-0.2431465E 

02 

395 

2490906E 

02 

396 

-0.2550349E 

02 

397  -0.2609792E 

02 

398 

-0.3056189E 

02 

399 

31 15636E 

02 

400 

-0.3410464E 

02 
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is  to  plot  an  engine  on  the  scatter  plot  every  time  a  new 
set  of  data  is  recorded,  under  the  assumption  that  when  the 
engine  leaves  the  overhaul  facility,  it  is  a  healthy  engine 
and  as long  as  its  position  on  the  scatter  plot  remains  approxi¬ 
mately  the  same,  we  assume  it  continues  to  be  a  healthy  engine. 
When  it  begins  to  deviate  significantly  from  the  position, 

especially  when  this  deviation  is  rapid,  one  should  suspect 
that  there  is  something  unusual  about  the  engine.  If  the 
deviation  is  also  toward  know  failures,  it  is  probably  due 
to  an  impending  failure.  In  these  cases,  extra  effort  should 
be  made  to  diagnose  this  engine  in  the  event  that  the  automated 
classifiers  have  not  already  identified  it.  One  action  that 
could  be  taken  if  one  were  to  use  this  technique  and  observe 
the  engine  deviating  from  its  normal  position  would  be  to 
increase  the  frequency  of  oil  samplings  for  that  engine. 

Clearly,  the  scatter  plot  is  not  a  tool  for  the  "sailor". 

It  is  laboratory  tool  which  can  be  used  by  the  more  experienced 
laboratory  technicians.  It  is  not  proposed  a  substitute  for 
the  automatic  detection  of  incipient  engine  failures.  The 
performances  presented  in  this  report  do  not  include  any  enhance¬ 
ment  which  could  be  achieved  by  the  use  of  the  scatter  plot 
approach.  It  is  offered  as  an  additional  benefit  which  can 
be  achieved  if  the  ADAPT  approach  to  deriving  automated  algorithms 
is  incorporated  and  applied  by  qualified  personnel.  There  is 
no  additional  cost  for  including  this  capability  since  the 
software  is  already  required  and  the  presentation  of  the  informa¬ 
tion  is  merely  the  addition  of  a  few  printing  instructions. 
Although  the  performances  that  are  quoted  in  this  report  do 
not  include  any  contribution  that  could  be  achieved  by  use  of 
these  scatter  plots,  examples  such  as  those  shown  in  Figure  19 
suggests  that  a  significant  increase  in  performance  is  possible 
using  these  diagnostic  aids.  If  effectively  applied  it  is 
likely  they  could  virtually  eliminate  any  preflight  and  inflight 
engine  aborts.  It  is  recommended  that  this  technique  be  investi¬ 
gated  as  part  of  the  Phase  II  program  and  that  its  impact  on 
both  the  cost  of  training^  performing  the  analysis  and  the  quality 
of  the  analysis  be  developed  and  reported  as  part  of  any  follow 
on  Phase  II  studies. 

4.3  IDENTIFICATION  OF  FAILURES 

In  the  preceding  Section  4.2,  we  illustrated  two  algorithms 
for  detecting  unhealthy  engines  as  well  as  an  approach  for  using 
the  ADAPT  scatter  plots  for  identifying  unhealthy  engines.  In 
this  section,  we  shall  discuss  two  algorithms  for  determining 
the  most  likely  cause  of  the  incipient  failure  and  which  are 
suitable  and  proposed  for  implementation  in  the  automated  system. 
We  shall  also  indicate  an  approach  that  may  be  used  with  the 
unique  ADAPT  analysis  tools  by  experienced  laboratory  techni¬ 
cians  or  engine  mechanics  to  assist  in  diagnoses  of  the  engine's 
problems. 
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The  two  types  of  algorithms  which  have  been  developed 
to  illustrate  the  diagnoses  of  an  engine  failure  are  the 
failure  detection  and  the  failure  classification  algorithms. 
These  algorithms  although  conceptually  very  similar,  often 
are  quite  different  in  both  their  physical  basis  for  working 
and  their  performance.  The  failure  detection  algorithm  is  an 
algorithm  which  is  developed  by  finding  the  best  separation 
between  a  specific  failure  type  and  all  other  engines  in¬ 
cluding  both  failed  and  unfailed  engines.  Thus,  this  algorithm 
both  detects  and  identifies  this  type  of  failure.  The 
alternative,  the  failure  classification  algorithm,  is  an 
algorithm  which  is  derived  to  separate  a  specific  type  of 
failure  from  all  other  failures.  It  is  derived  using  only 
histories  for  engines  in  which  an  incipient  failure  is 
expected.  Although  intuitively,  one  might  expect  this  latter 
approach  would  provide  somewhat  better  performance  our 
experience  has  shown  that  this  is  not  always  the  case  and 
both  types  of  algorithms  should  be  investigated. 

Oil  Failure  Detection  Algorithm 

To  illustrate  the  failure  detection  algorithm,  we  have 
taken  the  same  data  that  was  used  to  derive  the  incipient 
failure  detection  algorithm  and  derived  an  algorithm  for 
detecting  engine  oil  problems.  The  results  for  this  algorithm 
are  presented  in  the  same  format  as  the  previous  algorithms 
presented  in  Figures  14  and  16.  The  corresponding  figures 
for  the  oil  failure  detection  algorithm  are  Figures  20  and 
21.  Tables  7  and  8,  are  an  alternate  presentation  of  the 
same  information. 

Figure  20  shows  the  detection  statistic  resulting  from 
the  application  of  the  oil  system  failure  detection  algorithm 
to  the  entire  590  cases  used  in  this  study.  The  various 
failures  are  identified  as  before  by  the  arrows  along  the  top 
of  the  figure  and  the  engine  by  the  arrows  along  the  bottom 
of  the  figure.  Those  failures  which  could  be  positively 
identified  from  either  the  3M  or  NOAP  data  as  oil  system 
failures  are  included  in  solid  facing  triangles.  They  are 
failures  F3,  F4  and  F6.  The  failure  mode  for  the  three 
failures  included  in  dashed  triangles  could  not  be  determined 
from  the  available  3M  and  NOAP  data.  For  this  reason,  we  must 
assume  that  they  could  be  either  oil  system  failures  or  non¬ 
oil  system  failures.  The  training  was  based  on  assuming  that 
F3,  F4  and  F6  were  oil  system  failures  and  omitting  F8,  F10 
and  Fll  from  the  derivation  of  the  algorithm. 
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TABLE  7 

DETECTION  STATISTIC  FOR  DETECTION  OF  ENGINE  OIL  PROBLEMS  (REVISED) 

HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


1  -Q.4521710E  02 
S  -O.44910O0E  02 
9  -0.4463609E  02 
13  -0.4440924E  02 
17  -O.5699300E  02 
21  -0.6493423E  02 
iS  -0.4909705E  02 
29  -0. 524926 3E  02 
33  -0.5172043E  02 
37  -0.4600749E  02 
41  -0.S037225E  02 
4S  -0.4400455E  02 
49  -0.2017331E  02 
53  -0.3290410E  02 
57  -O.3233S0OE  02 
41  -0.3748947E  02 
45  -0.3710939E  02 
49  -0.3629161E  02 
73  -0.9090031E  01 
77  -0.9009360E  01 
•1  -0.4089635E  01 
05  -0 . 7224070E  01 
09  0.4551005E  01 
93  0.4449545E  01 
97  -0.1909379E  00 
101  -0.5919573E  01 
105  -0.5997841E  00 
109  -0.1309193E  02 
113  -0.1741542E  02 
117  -0.1704417E  02 
121  -0.1632649E  02 
125  -0.1590939E  02 
129  -0. 157 >677E  02 
133  -0.1540751E  02 
137  -0.1507826E  02 
141  -0.1474902E  02 
145  -0.134215OE  02 
149  - 0.530 94 06 E  02 
153  -0.5413574E  02 
157  -0.5459404E  02 
161  -0.5529570E  02 
165  -0.5702281E  02 
169  -0.5773491E  02 
173  -O.5910175E  02 
177  -0.5936124E  02 
101  -0.6307039E  02 
105  -O.6190778E  02 
109  -0.6131206E  02 
193  -0.6127024E  02 
197  -0.6124445E  02 


2  -0.4515012E  02 
6  -0.4403076E  02 
10  -0.4459941E  02 
14  -0.4445259E  02 
10  -0.5579056E  02 
22  -0.6643373E  02 
26  -0.5064605E  02 
30  -0.5040923E  02 
34  -0.5178052E  02 
30  -0.5464900E  02 
42  -0.4679459E  02 
46  -0. 1534027E  02 
50  -0.31 12984E  02 
54  -0.3309274E  02 
58  -0.3144016E  02 
62  -0.3755495E  02 
66  -0.3693959E  02 
70  -0.3605009E  02 
74  -0.9075890E  01 
70  -0.0987170E  01 
02  -0.5051376E  01 
86  -0.5246345E  01 
90  0. 1901726E  02 

94  0.6730375E  00 

90  -0.2900615E  01 
102  -0.6044024E  01 
106  -0.7429031E  01 
110  -0.1570429E  02 
114  -0.1739004E  02 
118  -0.1692961E  02 
122  -0. 1612544E  02 
126  -0.1500094E  02 
130  -0. 1565442E  02 
134  -0.1532520E  02 
130  -0 . 1499598E  02 
142  -0. 14666 74E  02 
146  -0.1540591E  02 
150  -0.5394770E  02 
154  -0.5471291E  02 
158  -0 .54 72 305 E  02 
162  -0.55180UE  02 
166  -0.5709251E  02 
170  -0.5054370E  02 
174  -0.5922661E  02 
170  -0.5940610E  02 
182  -0.6393064E  02 
106  -0.6133737E  02 
190  -0.6130356E  02 
194  -0.6126979E  02 
190  -0.6123596E  02 


3  -0. 45081 39E  02 
7  -0.4476709E  02 
11  -0.4456265E  02 
15  -0.4441509E  02 
19  -0.5734660E  02 
23  -0.6092596E  02 
27  -0.5069073E  02 
31  -0.4960000E  02 
35  -0.5015779E  02 
39  -0.5485431E  02 
43  -0.4696693E  02 
47  -0.19O3943E  02 
51  -0. 3276608E  02 
55  -0. 33391 10E  02 
59  -0.2975473E  02 
63  -0.3742020E  02 
67  -0.3676902E  02 
71  -0.2672935E  02 
75  -0.9053735E  01 
79  -0.0322065E  01 
03  -0.5905597E  01 
07  0.2070719E  01 
91  O.1999110E  02 

95  -0. 361156 7F  01 
99  -0.3043306E  01 
103  -0.6140765E  01 
107  -0.9258041E  01 
111  -0.1716467E  02 
115  -0.1736609E  02 
119  -0. 1672050E  02 
123  -0. 1595024E  02 
127  -0. 1 586845 E  02 
131  -0.15572UE  02 
135  -0.15242B7E  02 
139  -0.1491364E  02 
143  -0.1458439E  02 
147  -0.5370664E  02 
151  -0.5395934E  02 
155  -0.5482105E  02 
159  -0.5407923E  02 
163  -0.5620393E  02 
167  -0.5716219E  02 
171  -0.5941479E  02 
175  -0.5927145E  02 
179  -0.5945096E  02 
103  -0.6320033E  02 
107  -0.6132093E  02 
191  -0.6129512E  02 
195  -0.6126132E  02 
199  -0.612275SE  02 


4  -0.4499973E  02 
0  -0.4469542E  02 
12  -0.4452599E  02 
16  -0.5707340E  02 
20  -0.6231561E  02 
24  -0.5216043E  02 
20  -0.5342007E  02 
32  -0.5047729E  02 
36  -0.4900490E  02 
40  -0.5252792E  02 
44  -0.4705336E  02 
48  -0.2521664E  02 
52  -0 . 3207547E  02 
56  -0.3323145E  02 
60  -0.2949234E  02 
64  -0.3727917E  02 
68  -0.3653310E  02 
72  -0.912021 IE  01 
76  -0.9031554E  01 
00  -0 . 7650640E  01 
04  -0.6713437E  01 
00  0.3762153E  01 

92  0. 1129259E  02 

96  0.7018585E  00 

100  -0.573323OE  01 
104  -0.3331629E  01 
100  -0. 110670 IE  02 
112  -0.1744030E  02 
116  -0.1745705E  02 
120  -0. 165275 3E  02 
124  -0. 1592976E  02 
120  -0. 1579106E  02 
132  -0.1548981E  02 
136  -0.1516Q59E  02 
140  -0.1403134E  02 
144  -0.1431020E  02 
148  -0.5304032E  02 
152  -0.5394977E  02 
156  -0.5446591E  02 
160  -0.5500751E  02 
164  -0.56 9531 3E  02 
168  -0.5744955E  02 
172  -0.5913690E  02 
176  -0.5931635E  02 
100  -0.5949506E  02 
184  -0.6263806E  02 
180  -0.6132050E  02 
192  -0.6120667E  02 
196  -0.6125280E  02 
200  -0.6121907E  02 
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TABLE  7 

DETECTION  STATISTIC  FOB  DETECTION  OF 

CONTINUED 

HOI  DETECTION  STAT  HOL  DETECTION  STAT 

201  -0.6121062E  02  202  -0.6120216E  02 

20$  -0.6117685E  02  200  -0.6U6840E  02 

200  -0.6114307E  02  210  -0.6U3461E  02 

213  -0.6110925E  02  214  -0.6U0080E  02 

217  -0.59051B2E  02  210  -0.5902646E  02 

221  -0.5895042E  02  222  -0.5892508E  02 

22$  -O.S004900E  02  226  -0. 70706 19E  02 

229  -0.6007796E  02  230  -0.6825630E  02 

233  -0.6639133E  02  234  -0.6S76970E  02 

237  -0.6390472E  02  230  -0.6320311E  02 

241  -0.6124360E  02  242  -0.612176SE  02 

245  -0.6113943E  02  246  -0.61U336E  02 

249  -0.6140051E  02  2S0  -0.6149620E  02 

253  -0.75U588E  02  254  -0.7494720E  02 

257  -0.7444U2E  02  250  -0.7427237E  02 

261  -0.6060962E  02  262  -0.6374940E  02 

265  -0.6456500E  02  266  -0.6400262E  02 

269  -0.6366039E  02  270  -0.6365705E  02 

273  -0.2646526E  02  274  -0.2636403E  02 

277  -0.3961717E  02  270  -0.3977402E  02 

201  -0.4024454E  02  202  -0.4040135E  02 

205  -0.4070430E  02  206  -0.4005060E  02 

209  -0.4104976E  02  290  -0.411U40E  02 

293  -0.3340411E  02  294  -0.3345348E  02 

297  -0.3357726E  02  290  -0.3346909E  02 

301  -0.1628125E  02  302  -0.151O403E  02 

305  -0.1077140E  02  306  -0.1066073E  02 

309  -0.0215460E  01  310  -Q.0U2670E  01 

313  -O.0O96300E  02  314  -0.0110204E  02 

317  -0.9023306E  02  310  -0.9043175E  02 

321  -0.9100620E  02  322  -0.9130432E  02 

325  -0.9620345E  02  326  -0.9617155E  02 

329  -0. 96076 05E  02  330  -0.9604425E  02 

333  -0.6106972E  02  334  -0.6104552E  02 

337  -O.6417204E  02  330  -0.6494061E  02 

341  -0.6673294E  02  342  -O.66O00UE  02 

345  -0.6604039E  02  346  -0.6665344E  02 
349  -0.6609254E  02  350  -0.6590555E  02 
353  -0.6534470E  02  354  -0.4515771E  02 
357  -0.6423610E  02  350  -0.6419990E  02 
361  -O.113O970E  03  362  -0.1135536E  03 
365  -0. 1149213E  03  366  -0.1153310E  03 
369  -0.1165636E  03  370  -0.U69743E  03 
373  -0. 1102061E  03  374  -O.1106I6OE  03 
377  -0.U96360E  03  370  -0.U93759E  03 
301  -0.0049367E  02  302  -0.0033293E  02 

305  -0.7950040E  02  306  -0.7910170E  02 

309  -0 . 7022570E  02  390  -0.7790710E  02 

393  -0.7695107E  02  394  -0.7663243E  02 

397  -0.7567636E  02  390  -0.5453011E  02 


ENGINE  OIL  PROBLEMS  (REVISED) 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

203  -0.6U9374E  02  204  -0.6U8527E  02 
207  -0.61I5993E  02  208  -0.6115150E  02 
211  -0.6U2614E  02  212  -0.6U1769E  02 
215  -0.6109236E  02  216  -0.5907713E  02 
219  -0.59001 13E  02  220  -0.5897577E  02 
223  -0.5889972E  02  224  -0.5887436E  02 
227  -0.7012132E  02  228  -0.6949969E  02 
231  -0.6763464E  02  232  -0.6701297E  02 
235  -0.6514806E  02  236  -0.6452641E  02 
239  -0.6266139E  02  240  -0.6203973E  02 
243  -0.6U9159E  02  244  -0.6116551E  02 
247  -0.6120908E  02  248  -0.6130479E  02 
251  -0.7502585E  02  252  -0.7507082E  02 
255  -0 . 7477852E  02  256  -0.7460983E  02 
259  -0.7410371E  02  260  -0.6051379E  02 
263  -0.6409325E  02  264  -0.6443701E  02 
267  -0.6366547E  02  268  -0.6366292E  02 
271  -0.2666772E  02  272  -0.2656647E  02 
275  -0.2626303E  02  276  -0.2616193E  02 
279  -0.3993089E  02  280  -0.4008772E  02 
283  -0.4055818E  02  284  -0.4071497E  02 
207  -0.4091702E  02  280  -0.4098343E  02 
291  -0. 4098241 E  02  292  -0.3334B83E  02 
295  -0.3350285E  02  296  -0.3355225E  02 
299  -0.3422375E  02  300  -0.3498473E  02 
303  -0. 1379288E  02  304  -0.1228214E  02 
307  -0. 1056603E  02  308  -0.1046339E  02 
311  -0.8052750E  02  312  -0.8074570E  02 
315  -0.8140021E  02  316  -0.8776804E  02 
319  -0.90649B4E  02  320  -0.9086807E  02 
323  -0.9152251E  02  324  -0.9174069E  02 
327  -0.9613971E  02  320  -0.9610789E  02 
331  -0.5911107E  02  332  -0.6012444E  02 
335  -0.6262129E  02  336  -0.6339709E  02 
339  -0.6572441E  02  340  -0.6650010E  02 
343  -0.6704326E  02  344  -0.6702736E  02 
347  -0.6646645E  02  340  -0. 6627951 E  02 
351  -0.6571060E  02  352  -0.6553166E  02 
355  -0.6475597E  02  356  -0.6427237E  02 
359  -0.6416300E  02  360  -0.6412762E  02 
363  -0.1140094E  03  364  -0.1144651E  03 
367  -0.U57425E  03  360  -0.1161532E  03 
371  -0.1 17304 9E  03  372  -0.U77955E  03 
375  -0. 1190273E  03  376  -0.1194380E  03 
379  -0. 1191159E  03  300  -0.U00558E  03 
303  -0.0013596E  02  384  -0.7981914E  02 
387  -0.7886307E  02  388  -0.7854453E  02 
391  -0.7758836E  02  392  -0.7726978E  02 
395  -0.7631372E  02  396  -0.7599506E  02 
399  -0.5421155E  02  400  -0.5661084E  02 
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TABLE  7 

DETECTION  STATISTIC  FOR  DETECTION  OF 

CONTINUED 

HOL  DETECTION  STAT  HOL  DETECTION  STAT 

401  -0.565207BE  02  402  -0.S643063E  02 

405  -0.542501 IE  02  406  -0.5415996E  02 
409  -0.6370996E  02  410  -0.6354S72E  02 

413  -0.6363123E  02  414  -0.6377501E  02 

417  -0.6411783E  02  410  -0.613U37E  02 

421  -0.9544771E  02  422  -O.S4O5320E  02 

425  -0.8341273E  02  426  -O.S319920E  02 

429  -O.0255073E  02  430  -0.8234523E  02 

433  -0.5I70465E  02  434  -0.6149120E  02 

437  -O.8O05O63E  02  438  -0.5980956E  02 

441  -0.9262015E  02  442  -0.8097754E  02 

445  -0.8033699E  02  446  -0.8012355E  02 

449  -0.7946302E  02  450  -0.7926953E  02 

453  -0.7862895E  02  454  -0.7841550E  02 
457  -0.7777489E  02  458  -0. 5673381 E  02 
461  -0.4958972E  02  462  -0.6216179E  02 
465  -0.5978737E  02  466  -0.5899585E  02 
469  -0.5662128E  02  470  -0.5550555E  02 
473  -0.5509776E  02  474  -0.5496184E  02 
477  -0.5483234E  02  470  -0.5480078E  02 
481  0.5168835E  02  482  0.5151392E  02 

485  0.5099222E  02  486  0.5081784E  02 

489  0.5029535E  02  490  0.5012173E  02 

493  0.4959927E  02  494  0.4942561E  02 
497  0.4890315E  02  498  0.4872873E  02 
501  0.4832040E  02  502  O.40I8338E  02 

505  0.4777274E  02  506  0.4727013E  02 

509  0.4762689E  02  510  0.4774586E  02 

513  0.4507271E  02  514  0.44970I2E  02 
517  0.4529344E  02  510  0.4SS8540E  02 
521  0.4646117E  02  522  0.4675307E  02 

525  0.4762880E  02  526  0.4792072E  02 

529  0.4850961E  02  530  0.4851689E  02 

533  0.4853857E  02  534  O.40545S7E  02 
537  0.4856755E  02  538  0.4857483E  02 

541  0.4859656E  02  542  0.4860381E  02 

545  0.4862553E  02  546  0.4863277E  02 

549  0.4865451E  02  550  0.4866176E  02 

553  -0.6339880E  02  554  -0.6339613E  02 
557  -0.6338826E  02  550  -0.6338562E  02 
561  -0.6339072E  02  562  -0.6339290E  02 
565  -O.6339930E  02  566  -0.5275635E  02 
569  -0.5731487E  02  570  -0.5721040E  02 

573  -0.5670541E  02  574  -0.5649437E  02 
577  -0.6591249E  02  578  -0.6585834E  02 

501  -0.6584662E  02  582  -0.6583974E  02 
585  -0.6581918E  02  586  -0.6581230E  02 
509  -0.6579175E  02  590  -0.6578488E  02 


ENGINE  OIL  PROBLEMS  (REVISED) 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

403  -0.5634045E  02  404  -0.5625027E  02 
407  -0.6470805E  02  408  -0.6429916E  02 
411  -0.6338760E  02  412  -0.6348744E  02 
415  -0.6395575E  02  416  -0.6410516E  02 
419  -0.5974738E  02  420  -0.6052890E  02 
423  -0.8383977E  02  424  -0.8362627E  02 
427  -0.8298572E  02  428  -0.8277225E  02 
431  -0.8213170E  02  432  -O.8191021E  02 
435  -0.8127766E  02  436  -0.8106415E  02 
439  -0.5959605E  02  440  -0.6227103E  02 
443  -0.8076402E  02  444  -0.8055052E  02 
447  -0. 7990997E  02  448  -0.7969655E  02 
451  -0.7905597E  02  452  -0.7884251E  02 
455  -0.7620195E  02  456  -0.7798845E  02 
459  -0.5652034E  02  460  -0.5919531E  02 
463  -0.6137035E  02  464  -0.60578B6E  02 
467  -0.5820430E  02  468  -O.5741201E  02 
471  -0.5536963E  02  472  -0.5523369E  02 
475  -0.5489S48E  02  476  -0.5486389E  02 
479  -0.5476923E  02  400  -0.5473767E  02 
483  0.5134027E  02  484  O.5U6S80E  02 
487  0.5064418E  02  488  0.5046974E  02 
491  0.4994731E  02  492  0.4977365E  02 
495  0.4925117E  02  496  0.4907758E  02 
499  0.4859407E  02  500  0.4845724E  02 
503  0.4804654E  02  504  9.4790970E  02 
507  0.4738904E  02  508  0.4750797E  02 
511  0.4527795E  02  512  0.4517535E  02 
515  0.4486754E  02  516  0.45Q0160E  02 
519  0.4587735E  02  520  0.4616927E  02 
523  0.4704500E  02  524  0. 4733691 E  02 
527  0.4821252E  02  528  0.4850458E  02 
531  0.4852406E  02  532  0.4853136E  02 
535  0. 485530 7E  02  536  0.4856033E  02 
539  0.4858209E  02  540  0.4858932E  02 
543  0.486  U04E  02  544  0.4861826E  02 
547  0.4864001E  02  548  0.4864725E  02 
551  -0.6340410E  02  552  -0.6340147E  02 
555  -0.6339352E  02  556  -0.6339087E  02 
559  -0.6338643E  02  560  -0.6338860E  02 
563  -0.6339508E  02  564  -0.6339720E  02 
567  -0.5752370E  02  560  -0.5741928E  02 
571  -0.5710603E  02  572  -0.5691624E  02 
575  -O.5620340E  02  576  -0.6596666E  02 
579  -0. 6506031 E  02  500  -0.6585347E  02 
503  -0.6583287E  02  584  -0.6582602E  02 
587  -0.6580545E  02  500  -0.6579858E  02 
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TABLE  8 

V  RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  DETECTING  ENGINE  OIL  PROBLEMS 


MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 
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TABLE  8 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  DETECTING  ENGINE  OIL  PROBLEMS 

CONTINUED 

MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 
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However,  Figure  20  shows  the  results  of  applying  the 
algorithm  to  these  three  failure  types.  The  algorithm  would 
indicate  that  failure  F8  was  not  a  failure  in  the  oil  system 
whereas  failures  10  and  11  were  failures  of  the  oil  system. 

Once  again  for  a  detail  description  of  what  each  of  these 
failures  are  the  reader  is  referred  to  Table  2.  Tables  7  and  8 
present  the  same  information  as  Figures  20  and  21  in  more 
detail  so  that  a  more  careful  analysis  is  possible  by  inspecting 
the  figure. 

Figure  21  shows  the  relative  importance  vector  for  the 
detection  of  engines  with  oil  problems.  The  general  shape 
of  this  curve  is  very  similar  to  the  incipient  failure 
detection  algorithm  (Figure  16)  except  that  the  test  cell 
measurements  have  become  relatively  more  important  then  they 
were  for  the  detection  of  incipient  failures.  This  is  primarily 
because  the  comparison  between  the  reference  test  cell  per¬ 
formance  and  the  observed  test  cell  values  have  become  very 
significant.  Although  there  are  many  wear  metals  which  are 
important  to  both  algorithms,  there  are  elements  which  are 
important  and  differ  between  the  two  algorithms.  It  is  in¬ 
teresting  to  note  that  4  of  the  5  most  important  wear  metals 
for  this  algorithm  (Na,  Cd,  Mn,  V  and  Mg)  are  not  used  in  the 
present  NOAP  program.  The  details  of  this  relative  importance 
vector  are  also  presented  in  Table  8  so  that  the  reader  may 
perform  his  own  analysis. 

Oil  Failure  Classification  Algorithm 

As  previously  discussed,  the  classification  algorithms 
are  developed  using  only  the  failure  cases.  Figure  22  and 
Table  9  present  the  detection  statistic  for  separating  those 
failures  which  were  engine  oil  failures  from  all  of  the  other 
failures.  Once  again,  the  location  of  the  engine  oil  failure 
is  indicated  by  the  downward  facing  triangle,  the  solid  tri¬ 
angle,  for  those  which  were  known  from  the  truth  data  and  the 
dashed  triangles  indicating  those  for  which  the  truth  data 
did  not  define  whether  the  failure  was  in  the  oil  system  or  not. 
The  engine  has  also  been  identified  on  Figure  22.  The  only 
observations  incorrectly  identified  by  this  algorithm  are  the 
last  eight  observations  of  failure  F6.  However,  these  are  from 
the  same  engine  as  F7  and  represent  an  example  of  the  inter¬ 
polation  problem  discussed  in  Section  3.5.  Figure  23  and  Table 
10  present  the  corresponding  relative  importance  vector  for 
separating  engine  oil  failures.  It  differs  from  the  relative 
importance  vector  for  detecting  engine  oil  failures  both  in 
the  detailed  narrow  band  structure  and  in  that  the  oil  measure¬ 
ments  are  more  important  relative  to  the  test  cell  measurements 
then  they  were  for  the  oil  failure  detection  algorithm.  However, 
it  is  very  similar  to  the  relative  importance  vector  for  detect¬ 
ing  incipient  engine  failures. 


FIGURE  22  DETECTION  STATISTIC  FOR 
CLASSIFICATION  OF  ENGINE  OIL  PROBLEM 
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TABLE  -  9 

DETECTION  STATISTIC  FOR  CLASSIFICATION  OF  ENGINE  OIL  PROBLEM 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

1  -0.4415204E  02  2  -0.4399709E  02 

5  -0.4712364E  02  6  -0.4638564E  02 

9  -0.44B7460E  02  10  -0.4512S08E  02 

13  -0.399702 IE  02  14  -0.3807492E  02 

17  -0. 506486 9E  02  16  -0.5073520E  02 

21  -0.5025737E  02  22  -0.5008076E  02 

25  -0.4947662E  02  26  0.1257971E  02 

29  0.9747601E  01  30  0.8857703E  01 

33  0.5020659E  01  34  0.7020608E  01 

37  0.5617196E  01  38  0.2694096E  01 

41  -0. 1501146E  01  42  0.1083861E  02 

45  0.1616147E  02  46  0.1308310E  02 

49  0.8854174E  01  50  0.1U6453E  02 

53  0.7326275E  00  54  -0.1967335E  01 

57  -0.6369537E  01  56  -0.6450432E  01 

61  -0. 1353581C  02  62  -0.1672888E  02 

65  -0.18S3061E  02  66  -0.1913669E  02 

69  -0. 1976271E  02  70  -0.1932545E  02 

73  -0. 1798196E  02  74  -0.1752245E  02 

77  -0. 1608897E  02  76  -0.1561114E  02 

81  -0.1417754E  02  82  -0.1369973E  02 

65  -0. 1226619E  02  86  -0.U76836E  02 

89  -0.1105416E  02  90  -0.1336581E  02 

93  -0.6783957E  02  94  -0.6783420E  02 

97  -0.6781786E  02  96  -0.6781248E  02 

101  -0.5594650E  01  102  -0.5353465E  01 

105  -0.4630578E  01  106  -0.4389922E  01 

109  -0.1920195E  02  110  -0.1824170E  02 

113  -0. 1536091E  02  114  -0.1440063E  02 

117  -0. 1654030E  02  118  -0.1725314E  02 

121  -0 . 2570987E  02  122  -0.4080232E  02 

125  -0.4365424E  02  126  -0.4460413E  02 

129  -0.4861197E  02  130  -0.5224924E  02 

133  -0.672791 IE  02  134  -0.6833910E  02 

137  -0.6900124E  02  138  -0.6880238E  02 

141  -0.6513737E  02  142  -0.6410930E  02 

145  -0.6139658E  02  146  -0.6053101E  02 

149  -0.5793428E  02  150  -0.5706873E  02 

153  -0.5646278E  02  154  -0.5646767E  02 

157  -0.5715614E  02  156  -0.5737912E  02 

161  -0.5604750E  02  162  -0.5627037E  02 

165  -0.5920906E  02  166  -0.598051 IE  02 

169  -0.5987633E  02  170  -0.5990005E  02 

173  0.3960297E  02  174  0.3911263E  02 

177  0.3764232E  02  178  0.3715204E  02 

181  0.3566100E  02  162  0.3519096E  02 

185  0.3372122E  02  186  0.3323195E  02 

189  0. 1067016E  02  190  0.1249387E  02 

193  0.7632660E  02  194  0.7651295E  02 

197  0.7707031E  02  196  0.7725664E  02 


HOL  DETECTION  STAT  HOL  DETECTION  STAT 

3  -0.4380255E  02  4  -0.4546309E  02 

7  -0.4568194E  02  6  -0.4537836E  02 

11  -0.4463634E  02  12  -0.4230336E  02 

15  -0 . 3850592E  02  16  -0.5096230E  02 

19  -0.5061043E  02  20  -0.5043391E  02 

23  -0.4988054E  02  24  -0.4967665E  02 

27  0. 1156945E  02  28  0.1064094E  02 

31  0. 9560899E  01  32  0.5922699E-01 

35  0.6747023E  01  36  0.6332526E  01 

39  -0. 1458354E  01  40  -0.4112986E  01 

43  0. 1452299E  02  44  0.1661815E  02 

47  0. 1042656E  02  48  0.7911560E  01 

51  0.6430694E  01  52  0.3581647E  01 

55  -0.3005142E  01  56  -0.3762299E  01 

59  -0. 1053133E  02  60  -0.1261234E  02 

63  -0. 1731636E  02  64  -0.1792444E  02 

67  -0. 1974267E  02  66  -0.2019997E  02 

71  -0. 1868817E  02  72  -0.1645096E  02 

75  -0. 1704463E  02  76  -0.1656682E  02 

79  -0.1513330E  02  80  -0.1465552E  02 

83  -0. 1322192E  02  84  -0.1274402E  02 

87  -0.1131052E  02  88  -0.1083266E  02 

91  -0.6785039E  02  92  -0.6784502E  02 

95  -0.6782874E  02  96  -0.6782338E  02 

99  -0.6780701E  02  100  -0.6780159E  02 

103  -0.5112233E  01  104  -0.4871122E  01 

107  -0.2112233E  02  108  -0.2016216E  02 

111  -0.1728149E  02  112  -0.1632121E  02 

115  -0.15U460E  02  116  -0.1582753E  02 

119  -0. 1796605E  02  120  -0.1774898E  02 

123  -0.4175459E  02  124  -0.4270453E  02 

127  -0.4541019E  02  128  -0.4478146E  02 

131  -0.5803378E  02  132  -0.6354755E  02 

135  -0.6939905E  02  136  -0.6920015E  02 

139  -0.5748453E  02  140  -0.5728539E  02 

143  -0.6312778E  02  144  -0.6226219E  02 

147  -0.5966545E  02  148  -0.5879987E  02 

151  -0.5680908E  02  152  -0.5663605E  02 

155  -0.5671059E  02  156  -0.5693335E  02 

159  -0.5760185E  02  160  -0.5782462E  02 

163  -0.5849316E  02  164  -0.5871599E  02 

167  -0.5982883E  02  168  -0.5985256E  02 

171  0 . 3088055E  02  172  0.4009312E  02 

175  0.3862250E  02  176  0.3813261E  02 

179  0.3666168E  02  180  0.3617137E  02 

183  0.3470076E  02  184  0.3421062E  02 

187  0. 3274258E  02  188  0.1136013E  02 

191  0.7595470E  02  192  0.7614105E  02 

195  0.7669646E  02  196  0.7688481E  02 

199  0.7744298E  02  200  0. 776284 9E  02 
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DETECTION  STATISTIC  FOR  CLASSIFICATION  OF  ENGINE  OIL  PROBLEM 
CONTINUED 


HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT  HOL  DETECTION  STAT 


201 

0.7781463E 

02 

202 

0.7800041E 

02 

203 

0.7818669E 

02 

204 

0.7837225E 

02 

205 

0. 785505 9E 

02 

206 

0.7874406E 

02 

207 

0 . 7893037E 

02 

208 

0.791 1676E 

02 

209 

0.7850928E 

02 

210 

0. 7790109E 

02 

211 

0 . 7729291E 

02 

212 

0 . 766842 3E 

02 

213 

0.7607605E 

02 

214 

0.7546762E 

02 

215 

0. 748591 3E 

02 

216 

0.6665387E 

02 

217 

0.6838640E 

02 

218 

0.6811667E 

02 

219 

0.6785144E 

02 

220 

0.6758394E 

02 

221 

0.7191789E 

02 

222 

0.6972888E 

02 

223 

0.6754002E 

02 

224 

0.65351 14E 

02 

225 

0.6316225E 

02 

226 

0.6244119E 

02 

227 

0. 6269931 E 

02 

228 

0.6295744E 

02 

229 

0.6321556E 

02 

230 

0.6347371E 

02 

231 

0.6373178E 

02 

232 

0.6398994E 

02 

233 

0.642481 IE 

02 

234 

0.6450612E 

02 

23S 

0.6476428E 

02 

236 

0.6502235E 

02 

237 

0.6528050E 

02 

238 

0.6553873E 

02 

239 

0.6553026E 

02 

240 

0.6552260E 

02 

241 

0.6551486E 

02 

242 

0 .655071 7E 

02 

243 

0.6549944E 

02 

244 

0.6549168E 

02 

245 

0.6548396E 

02 

246 

0.6547626E 

02 

247 

0.6546855E 

02 

248 

0.6546086E 

02 

249 

0.654531 IE 

02 

250 

0.6544539E 

02 

251 

0.6543770E 

02 

252 

0.6542998E 

02 

253 

0.6542223E 

02 

254 

0.6541454E 

02 

255 

0.6540678E 

02 

256 

0.6539906E 

02 

257 

0.6539137E 

02 

258 

0.6538365E 

02 

259 

0.6537S95E 

02 

260 

0.6536816E 

02 

261 

-0.5651573E 

02 

262 

-0.5642024E 

02 

263 

-0.5632478E 

02 

264 

-0.5622928E 

02 

265 

-0.5613382E 

02 

266 

-0.5603835E 

02 

267 

-0.5594284E 

02 

268 

-0.5584746E 

02 

269 

-0.5574092E 

02 

270 

-0.5563036E 

02 

271 

-0.5551979E 

02 

272 

-0.5540 92 3E 

02 

273 

-0.5529854E 

02 

274 

lu 

O' 

O' 

N 

<0 

8 

• 

O 

02 

275 

-0.5507741E 

02 

FIGURE  -  23  RELATIVE  IMPORTANCE 
VECTOR  FOR  ENGINE  OIL  PROBLEM  CLASSIFIER  (8) 
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TABLE  -  10 

*  RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  CLASSIF1NG  ENGINE  OIL  PROBLEM  (6) 

MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


0 

1 

0. 1708716E-01 

2 

0.0 

3 

0.9981477E-01 

4 

0.0 

S 

-0. 1327353E 

02 

6 

0.5231941E 

01 

7 

0 . 2083905E 

02 

6 

0. 1660902E 

02 

V 

-0.9599669E 

01 

to 

0 . 3676787E 

01 

11 

0.5731066E 

01 

12 

0. 1505355E 

01 

13 

0.9697376E 

01 

19 

-0.1633542E 

02 

15 

-0.2466566E 

01 

16 

0.2531926E 

01 

0 

17 

0. 1667396E 

01 

16 

0.2319975E 

01 

19 

0.3611209E 

01 

20 

-0.5972656E 

01 

21 

-0.6582720E 

01 

22 

-0.6661767E 

01 

23 

-0.7533530E 

01 

24 

-0. 1077066E 

02 

25 

0.9921543E 

01 

26 

-0.1704132E 

01 

27 

-0.3500730E 

01 

26 

-0.2107566E 

01 

29 

0.9398791E 

01 

30 

-0.5566525E-01 

31 

0.3230633E 

01 

32 

-0. 1446351E 

01 

0 

33 

0.1156534E 

01 

34 

-0.6695745E 

00 

35 

-0. 163131 IE 

01 

36 

-0. 167202 3E 

02 

37 

0. 1079762E 

01 

38 

0. 1371016E 

01 

39 

0.2015679E 

00 

40 

0.2239396E 

01 

91 

0. 3277232E 

01 

42 

0.2993741E 

01 

43 

0.434 36 92E 

01 

44 

0.3515962E 

01 

95 

0 . 3453679E 

01 

46 

0.3991420E 

01 

47 

0.5110133E 

01 

46 

-0.2423665E 

00 

0 

99 

0.9769525E 

00 

50 

0. 3572313E 

00 

51 

-0.1263263E 

01 

52 

-0. 1648952E 

02 

53 

0.3566096E 

01 

54 

0.3506756E 

01 

55 

0.2050207E 

01 

56 

0.2 16534 7E 

01 

57 

0.3299122E 

01 

58 

0. 26331 97E 

01 

59 

0.4176794E 

01 

60 

0.1919446E 

01 

*1 

-0.2597695E 

01 

62 

-0. 1498322E 

01 

63 

-0.3292877E 

01 

64 

0.3297016E 

01 

f 

95 

0.3999676E 

01 

66 

0. 3836926E 

01 

67 

0.4809642E 

01 

66 

-0. 1042076E 

02 

59 

-0.3926576E 

01 

70 

-0.5431447E 

01 

71 

-0.9716269E 

01 

72 

-0. 1 040285 E 

02 

73 

-0.4406094E 

01 

74 

- V.1593315E 

01 

75 

-0. 7813620E 

01 

76 

-0. 1824554E 

02 

77 

0.9916269E 

01 

76 

0.3233565E 

01 

79 

0.9967410E 

00 

60 

0. 1903036E 

02 

e 

61 

9.6090330E 

01 

62 

0.3930690E 

01 

63 

0.521 1027E 

01 

64 

0. 1976137E 

02 

85 

0.2327701E 

01 

86 

0.2589579E 

01 

67 

0. 488066 7E 

01 

66 

0. 1966520E 

02 

69 

0.2927950E 

01 

90 

0.2299007E 

01 

91 

0.4373613E 

01 

92 

-0.3154135E 

01 

93 

0.9392192E 

01 

99 

-0.2866487E 

01 

95 

-0.2287567E 

01 

96 

0.2023663E 

02 

/ 

97 

0. 1293610E 

02 

98 

0.7582270E 

01 

99 

0. 1227656E 

02 

100 

-0.446166 7E 

01 

101 

0. 1102930E 

01 

102 

0. 1374022E 

01 

103 

0.2304264E 

00 

104 

-0.7965203E 

01 

105 

-0.21 10197E 

00 

106 

0.2772326E 

00 

107 

0. 1169260E 

00 

106 

0. 3630861 E* 

•01 

109 

0.9015266E 

01 

110 

0.70051 16E 

01 

111 

0.7234431E 

01 

112 

-0. 33561 95E 

01 

0 

113 

0.6761992E 

01 

114 

0.5738296E 

01 

115 

0.4694577E 

01 

116 

0.1577459E 

01 

117 

-0.1173645E 

02 

116 

-0. 1245040E 

02 

119 

-0.64396 72E 

01 

120 

0.1658870E 

02 

121 

0.1009739E 

02 

122 

0.1691042E 

02 

123 

0 . 3336266E 

01 

124 

-0. 1355265E 

02 

125 

-0.1152129E 

02 

126 

-0.8689876E 

01 

127 

-0.9037701E 

01 

128 

0.9346660E 

01 

€ 

129 

0.6598920E 

01 

130 

0.2594640E 

01 

131 

0.5355586E 

01 

132 

0. 1649163E 

01 

133 

0.2396297E 

01 

134 

0. 1551441E 

01 

135 

0.2257163E 

01 

136 

0.546724 IE 

00 

137 

-0.2519962E 

01 

138 

-0. 106596 9E 

01 

139 

-0.4824002E 

01 

140 

-0. 1067566E 

02 

191 

-0.U29963E 

02 

142 

-0. 1179667E 

02 

143 

-0. 9245122E 

01 

144 

0.0 

€ 

195 

0.0 

146 

0.0 

147 

0.5107758E 

01 

146 

-0.5211606E 

00 

199 

0.2361656E 

01 

150 

-0.9800714E 

00 

151 

0.6776667E- 

152 

0.0 

153 

0.6369149E 

01 

154 

0. 3172613E 

01 

155 

0.4649916E 

01 

156 

0.2697002E 

01 

157 

0.2719304E 

01 

156 

0.3435630E 

01 

159 

0.3411662E 

01 

160 

-0.2250030E 

01 

0 

161 

-0.2947617E 

01 

162 

-0.1966653E 

00 

163 

0. 1547014E 

01 

164 

0.4469539E 

00 

165 

-0. 17981 76E 

01 

166 

-0. 1061596E 

01 

167 

-0.5665090E 

00 

166 

-0. 162566 7E 

02 

169 

0.9166578E 

01 

170 

0.3014155E 

01 

171 

0.4166429E 

01 

172 

0.4396465E 

00 

173 

0.6936656E 

00 

174 

0.7626763E 

00 

175 

0.1673313E 

01 

176 

0.3270420E 

01 

0 

177 

0.6150932E 

01 

178 

0.5565330E 

01 

179 

0.4328151E 

01 

160 

0.4064556E 

00 

161 

0.5933560E 

01 

182 

0.5567901E 

00 

163 

0.8507768E 

00 

164 

0.1103059E 

02 

185 

0.9729066E 

01 

186 

0.3518648E 

01 

167 

0. 1665273E 

02 

166 

0.5495694E 

01 

169 

0.6325615E 

01 

190 

0.4070928E 

01 

191 

0.9449975E 

01 

192 

0.9016546E- 

•01 

• 

193 

0.2711521E 

01 

194 

0. 1093399E 

00 

195 

-0.6225745E 

01 

196 

-0.2144470E 

01 

197 

0.2427573E 

00 

198 

0. 1933781E 

01 

199 

-0.6556689E 

00 

200 

-0. 585551 5E 

01 

c 
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TABLE  -  10 

RELATIVE  IMPORTANCE  OF  EACH  MEASUREMENT  FOR  CLASSIFINS  ENGINE  OIL  PROBLEH  (8) 

CONTINUE!) 


MEAS  REL, IMPORTANCE  MEAS  REL  IMPORTANCE  HEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


201 

0.2271880E 

01 

202 

-0.6072969E 

00 

203 

0. 1981501E 

01 

204 

-0.3253478E 

00 

205 

0. 1828316E 

01 

206 

0.3230826E 

00 

207 

-0. 1585448E 

01 

208 

-0. 1034432E 

02 

209 

-0.3984319E 

01 

210 

-0.9179000E 

01 

211 

-0.7975349E 

01 

212 

-0.9855249E 

01 

213 

-0.2818507E 

01 

214 

-0.3768105E 

01 

215 

-0.5143890E 

01 

216 

0 . 2854557E 

01 

217 

0.9876358E 

01 

218 

0. 7995544E 

01 

219 

0.4672799E 

01 

220 

0.2157086E 

01 

221 

0.3193971E 

01 

222 

0.4423285E 

01 

223 

0.4752962E 

01 

224 

-0.2344 54 7E 

02 

225 

-0.8156336E 

01 

226 

0.183586 IE 

01 

227 

0 . 4877690E 

01 

228 

0.1486761E 

01 

229 

0. 1487247E 

01 

230 

-0.7368483E 

00 

231 

0. 1529655E 

01 

232 

0. 1586833E 

02 

233 

0.199861 IE 

02 

234 

0. 7535907E 

01 

235 

0. 38601 34E 

01 

236 

-0.4157930E-01 

237 

0.8909622E 

01 

238 

0. 3781782E 

01 

239 

Q.4312179E 

01 

240 

0. 1411208E 

02 

241 

0. 1742984E 

02 

242 

0. 1383129E 

02 

243 

0. 1077316E 

02 

244 

0 . 3940552E 

01 

245 

0.4286760E 

01 

246 

0.4177462E 

01 

247 

-0.1333949E 

01 

248 

-0.4701575E 

01 

249 

-0. 1130825E 

01 

250 

-0.5656132E 

00 

251 

0. 1018335E 

00 

252 

0.3075089E 

01 

253 

-0. 1917751E 

01 

254 

-0.8387217E 

00 

255 

0.2173745E 

01 

256 

0.2078156E 

02 

257 

0.2071355E 

02 

258 

0. 1093999E 

02 

259 

0. 1046851E 

02 

260 

0. 1102424E 

02 

261 

0. 1014806E 

02 

262 

0.9774412E 

01 

263 

0.7464877E 

01 

264 

-0.2126799E 

02 

265 

-0. 1331017E 

02 

266 

-0.606 35 34E 

00 

267 

-0.5054341E 

01 

268 

-0.2610070E 

00 

269 

-0.29766 JOE 

01 

270 

-0.7531023E 

01 

271 

-0. 1516469E 

01 

272 

0. 1761241E 

01 

273 

0. 1014161E 

02 

274 

0.4330349E 

00 

275 

-0.4941651E 

00 

276 

-0. 1829984E 

01 

277 

-0. 1096126E 

01 

278 

0.1022642E 

01 

279 

0.0 

280 

0.0 

281 

-0. 1005101E 

02 

282 

0.0 

283 

0.3540853E 

01 

284 

-0.1971309E 

01 

285 

0.0 

286 

-0. 1489S88E 

01 

287 

-0.2752574E 

01 

288 

0.3839211E 

01 

289 

0.0 

290 

0.0 

291 

-0.5314449E 

01 

292 

0.0 

293 

0.0 

294 

-0 . 1489588E 

01 

295 

0.0 

296 

0.0 

297 

-0.2544846E 

01 

298 

-0. 92041 14E 

01 

299 

0.0 

300 

0.0 

301 

0.0 

302 

0.3540853E 

01 

303 

0.8639866E-01 

304 

0.0 

305 

-0.7399948E 

00 

306 

0.0 

307 

0.0 

308 

0.0 

309 

0.0 

310 

0.0 

311 

-0 . 1489588E 

01 

312 

-0.2752090E 

01 

313 

-0.127891 IE 

01 

314 

-0.9592043E 

00 

315 

0.0 

316 

317 

-0.2391494E 

01 

318 

-0.6452022E 

01 

319 

0.0 

320 

o!o 

321 

0.0 

322 

0.0 

323 

0. 65301 14E 

00 

324 

0.0 

325 

0.6127911E 

00 

326 

0.0 

327 

0.0 

328 

0.0 

329 

0.0 

330 

0.0 

331 

0.0 

332 

0.0 

333 

0.4119656E 

01 

334 

-0.9592043E 

00 

335 

0.0 

336 

0.0 

337 

-0.1364345E 

01 

338 

0.0 

339 

0.U74122E 

01 

340 

-0.6452022E 

01 

341 

-0.9154174E 

01 

342 

0.0 

343 

0.0 

344 

0.0 

345 

0.0 

346 

0.0 

347 

0.0 

348 

0.0 

349 

0.0 

350 

0.0 

351 

0.2571131E 

01 

352 

0.0 

353 

0.4119658E 

01 

354 

-0.6452022E 

01 

355 

-0.4220631E 

01 

356 

0.0 

357 

0.0 

358 

0.0 

359 

0.0 

360 

0.0 

361 

0.0 

362 

0.0 

363 

0.0 

364 

0.0 

365 

0.3176150E 

01 

366 

-0.5459530E 

01 

367 

0.4178589E 

01 

368 

0.4995828E 

01 

369 

-0.6767488E 

01 

370 

0.0 

371 

0.0 

372 

0.0 

373 

0.0 

374 

0.0 

375 

-0.4208036E 

01 

376 

-0.3742008E 

01 

377 

0.4178589E 

01 

378 

0. 1454979E 

01 

379 

-0.3821950E 

01 

380 

0.0 

381 

0.0 

382 

0.0 

383 

0.0 

384 

0.0 

385  -0.U78470E  Oi 


tee  of  ADAPT  Analysis  Techniques  for  Failure  Diagonsls 


The  techniques  which  we  will  discuss  in  this  subsection 
have  the  same  relation  to  the  detection  algorithms  which  we 
have  discussed  in  the  preceding  subsections  as  the  scatter 
plot  incipient  failure  technique  had  to  the  incipient  failure 
algorithms.  That  is,  these  techniques  require  an  esqperienced 
roechanic/technician  to  examine  detail  results  which  have  been 
processed  to  make  the  pertinent  information  more  visible. 

They  are  intended  as  a  supplement  to  the  automated  system  and 
are  available  at  negligible  additional  effort  as  part  of  the 
ADAPT  processing.  Once  again,  all  of  our  performance  estimates 
do  not  include  performance  that  could  be  obtained  as  a  result 
of  making  use  of  these  capabilities  which  will  be  included 
in  the  system.  The  two  approaches  which  can  be  used  to 
assist  in  the  diagnosis  are  examination  of  the  scatter  plot 
and  creation  of  an  appropriate  relative  importance  vector. 

The  examination  of  the  scatter  plot  is  very  similar  to 
that  which  was  discussed  in  Section  4.2  for  incipient,  failure 
detection.  The  major  difference  would  be  that  superimposed 
upon  the  scatter  plot  as  shown  in  Figure  19  would  be  the 
identification  of  all  other  failures  which  have  previously 
occurred}  in  the  history  of  that  engine  and  appropriately 
adjusted  failures  which  have  occurred  in  other  engines.  Thus, 
by  noting  how  the  engine  deviates  from  its  normal  position 
relative  to  how  engines  with  other  failures  have  deviated, 
one  gets  information  which  would  be  very  useful  in  diagnosing 
the  failure. 

A  second  diagnostic  tool  which  can  be  applied:  1)  when 
the  automated  system  detects  an  incipient  failure  and  is  unable 
to  make  a  satisfactory  diagnosis  using  either  the  specific 


1 


1 


I 

B 

h 

1 

1 

k 

s 


\ 

failure  detector  or  the  failure  classifier,  or  2)  an  un- 
explained  deviation  from  the  scatter  plot  is  observed  is 
analysis  of  the  relative  importance  vector.  To  obtain  the 
relative  importance  vector,  an  algorithm  is  developed  to 
separate  the  case  or  cases  which  exhibit  the  failure  indica¬ 
tion  from  all  the  normal  cases  of  that  engine  and  possibly 
other  engines.  When  this  algorithm  is  developed,  its  relative 
importance  vector  can  be  displayed  and  it  will  indicate  which 
of  the  measurements  were  most  responsible  for  the  deviation 
from  a  healthy  engine.  This  has  proved  very  useful  in  a 
number  of  other  applications  and  is  described  in  considerable 
detail  in  a  study*  of  a  diagnostic  system  for  the  Saturn 
booster.  The  final  report  on  that  study  describes  this  method 
in  detail  and  presents  an  illustration  of  its  use  in  determin¬ 
ing  failure  inodes  in  the  Saturn  booster. 


4.4  ESTIMATING  TIME  TO  FAILURE 

The  final  task  which  will  be  accomplished  by  the  automated 
system,  whenever  it  is  possible,  is  to  provide  an  estimate 
of  the  number  of  engine  hours  remaining  before  the  failure 
is  expected  to  occur.  This  is  accomplished  using  a  time  to 
failure  algorithm.  This  algorithm  is  derived  for  each 
failure  mode  for  which  sufficient  experience  is  available. 

In  the  present  set  of  data,  the  engine  failure  due  to  a 
clogged  sump  strainer  provided  a  long  history  of  the  data 
prior  to  the  occurrence  of  this  failure.  Thus,  a  regression 
equation  was  developed  between  the  data  history  and  the  engine 
hours  remaining  to  failure.  This  regression  equation  has  an 
expected  3-Sigma  error  of  18  engine  hours. 

Figure  24  presents  a  plot  of  the  results  obtained  by 
applying  this  regression  equation  using  the  group  out  in¬ 
dependent  test  method  of  Lackenbrach  *  The  ordinate  of  this 
plot  is  the  estimated  time  until  the  failure.  The  actual  time 
is  plotted  along  the  abscissa.  Thus,  any  points  lying  on  the 
diagonal  line  dividing  the  figure  into  equal  halves  will 
represent  a  perfect  estimate  since  the  estimated  and  actual 
values  are  identical.  The  distance  away  from  this  diagonal 
line  through  the  center  of  this  figure  is  a  measure  of  the 
size  of  the  error.  Two  additional  dash  lines  have  been  added 


(5)  Hunter,  H.E.;  "Implementation  and  Demonstration  of  ADAPT 
Empirical  Analysis  at  KSC",  ADAPT  Report  #76-3,  June  1976. 
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to  Figure  24  and  are  the  lines  representing  a  error  of  plus 
18  hours  and  minus  18  hours,  respectively.  Points  lying 
between  these  two  lines  have  less  than  18  hour  errors  in 
their  prediction  of  the  time  to  failure. 

Figure  25  shows  the  relative  importance  vector  for 
the  time  to  failure  of  sump  strainer  in  main  oil  pump.  Since 
the  test  cell  data  does  not  vary  over  the  time  to  failure,  the 
test  cell  measurements  will  not  contribute  and  the  relative 
importance  vector  only  shows  the  NOAP  and  3M  data.  The  least 
important  elements  were  aluminum  (Al)  and  vanadium  (V) . 

Copper  (Cu) ,  lead  (Pb),  molybdenum  (Mo),  manganese  (Mn)  and 
zinc  (Zn)  concentration  increased  as  time  to  failure  decreased, 
but  magnesium,  titanium,  tin,  and  cadmium  concentration  decrease 
as  time  to  failure  decreases. 

4.5  Comparison  of  Automated  and  Manual  Test  Cell  Data 

Since  both  manually  and  automatically  recorded  test  cell 
were  used  in  this  study,  it  is  desirable  to  determine  if  there 
is  a  significant  difference  between  these  two  sets  and  if  so, 
the  expected  impact  on  the  results  of  this  study.  To  accomplish 
this,  an  algorithm  was  developed  separating  the  three  engines 
for  which  the  automated  test  cell  data  was  used  from  the 
remaining  11  engines.  The  expected  probability  of  error  obtained 
was  0.1.  This  is  significant,  but  not  a  strong  enough  separation 
to  be  expected  to  cause  difficulties  in  the  present  study. 

The  probability  of  error  of  0.1  is  sufficiently  strong 
to  warrant  a  study  of  the  differences  between  the  two  data 
sets.  Thus,  the  relative  importance  vector  was  constructed 
and  is  shown  in  Figure  26.  It  is  both  interesting  and  important 
to  note  that  the  dominant  difference  is  in  the  NOAP  measurements 
and  not  the  test  cell.  This  suggests  that  very  little  of  the 
difference  is  due  to  the  method  of  recording  the  test  cell  data. 
The  dominant  difference  is  most  likely  to  some  other  factor 
effecting  these  engines  uniformly  and  only  incidentally  related 
to  the  test  cell.  For  example,  about  a  third  of  the  observations 
are  from  the  two  engines  for  which  early  (and,  therefore,  manually 
recorded)  data  was  available.  Thus,  in  general,  the  NOAP  data 
for  the  automatically  recorded  test  cell  cases  is  more  recent 
data.  Any  improvements  in  the  NOAP  data  acquisition  and  record¬ 
ing  would  thus  become  a  marginally  good  discriminants.  This 
explanation  is  also  consistent  with  the  data  presented  in  Figure 
26  which  shows  that  for  many  of  the  elements  the  significant 
feature  is  the  change  (either  positive  or  negative)  over  the 
most  recent  75  hours.  This  is  the  behavior  that  would  be 
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GURE  25  RELATIVE  IMPORTANCE 
FOR  EOR  TIME  TO  E  A I  LURE  ESTIMATE 
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TABLE  11 

RELATIVE  IMPORTANCE  VECTOR  FOR  TIME  TO  FAILURE  ESTIMATE 


MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


1  0.1811954E  02 

5  -0.Z207857E  02 
9  -0.2275140E  02 
13  0.8708888E  01 

17  0. 1029133E  02 

21  -0.569 951 9E  02 
25  -0.8255156E  01 
29  -0.1112361E  02 
33  -0.2501199E  02 
37  -0. 330441 9E  02 
41  0.2732307E  02 

45  0.4553671E  02 

49  0.4388660E  02 

53  0.3926891E  02 

57  0.2712988E  02 

61  -0.5872766E  01 
65  0. 4855981 E  02 

69  0.2767566E  02 
73  0.7429203E  01 

77  0.5728101E  02 

81  -0.7346181E  00 
85  0.3591031E  02 

89  0.2139992E  02 

93  -0.3238458E  02 
97  0. 1539237E  02 

101  -0.3325U3E  02 
105  0. 1440086E  00 

109  -0.9679238E  01 
113  -0.4742670E  01 
117  -0. 1346525E  02 
121  0.1181895E  02 

125  0.6283908E  00 

129  0. 1211239E  01 

133  -0.2472729E  02 
137  -0.1818951E  02 
141  -0.2083104E  02 
145  0.0 

149  0.2118896E  01 

153  -0.5246964E  01 
157  -0.2099193E  02 
161  -0.4798971E  01 
165  0.8183815E  01 

169  0.305I796E  02 

173  -0.2077245E  02 
177  0.1782309E  02 

181  -0. 1281576E  02 
185  -0.4728918E  02 
189  -0.2350948E  02 
193  0.2168051E  02 

197  -0.5198413E  01 


2 

0.0 

6 

0.7134973E 

01 

10 

0.251 1470E 

02 

14 

0.2556085E 

02 

18 

0.8190779E 

01 

22 

-0.5497716E 

02 

26 

0. 1590917E 

02 

30 

-0.2102077E 

02 

34 

-0.2438673E 

02 

38 

-0.3725731E 

02 

42 

0.3327876E 

02 

46 

0.4269647E 

02 

50 

0.4388602E 

02 

54 

0. 1C24648E 

02 

58 

0.3277260E 

02 

62 

-0.1031424E 

02 

66 

0.4414424E 

02 

70 

0.2097910E 

02 

74 

0.3880087E 

02 

78 

0 . 1879800E 

02 

82 

0.9931237E 

01 

86 

0.4282072E 

02 

90 

0. 3102498E 

02 

94 

0.2133072E 

02 

98 

0.3889493E 

02 

102 

-0.3701300E 

02 

106 

-0.2574954E 

01 

110 

-0.1730185E 

02 

114 

0. 1253045E 

02 

118 

-0 . 1632515E 

02 

122 

0.2674503E 

01 

126 

0. 1006909E 

01 

130 

-0.2916342E 

02 

134 

-0.2404926E 

02 

138 

-0.2431472E 

01 

142 

-0.4296099E 

01 

146 

0.0 

150 

0. 1402496E 

01 

154 

-0.2250687E 

02 

158 

-0.2700819E 

02 

162 

-0. 3439010E 

01 

166 

0.1176498E 

02 

170 

-0.4269599E 

01 

174 

-0 . 2517036E 

02 

178 

0.2808128E 

02 

182 

-0. 1567142E 

02 

186 

-0. 1067442E 

02 

190 

-0 . 177091 IE 

02 

194 

0.4945058E 

01 

198 

•0.6124869E 

01 

3  0.3024832E  02 

7  -0.1971641E  01 
11  -0.2132210E  02 
15  0.3311343E  02 

19  0. 1011414E  02 

23  -0.5537741E  02 
27  0.7236737E  00 

31  -0.1879806E  02 
35  -0.2689346E  02 
39  -0.4412831E  02 
43  0.3153551E  02 

47  0.4175151E  02 
51  0.4479907E  02 

55  0. 1587686E  02 

59  0.3150589E  02 

63  -0.1315983E  02 
67  0.4324596E  02 

71  0.2863246E  02 

75  0.2411200E  02 

79  0.1647873E  02 
83  0. 1048442E  02 
87  0 . 3837354E  02 
91  0.3080017E  02 

95  0.7984145E  01 
99  0.3201576E  02 
103  -0.4401697E  02 
107  0.2327294E  01 
111  -C.1554684E  02 
115  0. 1482786E  02 
119  -0.1378180E  02 
123  0. 221432 7E  01 

127  0.2321857E  01 
131  -0.4761097E  00 
135  -Q.1285520E  02 
139  -0.1399133E  02 
143  -0. 1057266E  02 
147  -0.1687626E  01 
151  0.3101062E  00 

155  -0.5785944E  01 
159  -0.1623752E  02 
163  -0. 1021889E  02 
167  0.4533921E  01 
171  0.6792989E  01 

175  -0.3481260E  01 
179  0.1103659E  02 

183  -0.2136501E  02 
187  -0.2247888E  02 
191  -0. 1037648E  02 
195  0.4646144E  01 

199  -0.8457076E  01 


4  0.0 

8  -0.2326353E  02 
12  -0.2714983E  02 
16  0.5397070E  01 

20  -0.5144060E  02 
24  -0.1213946E  02 
28  -0.4766697E  02 
32  -0.3098183E  02 
36  0. 1109906E  02 

40  0. 3473918E  02 

44  0. 1390596E  02 

48  0.1975603E  02 

52  0.3373638E  01 

56  0.3435889E  02 

60  -0.2146 735E  02 
64  0.1297806E  02 

68  0.9972462E  01 

72  0.8075886E  01 

76  0.3086404E  01 

80  -0. 1646494E  02 
84  0. 1719724E  01 

88  0 . 2425583E  02 

92  0.2906693E  01 

96  0. 1009389E  01 

100  0. 1155224E  02 

104  0. 1580340E  02 

108  -0. 1105329E  02 
112  -0 . 2744084E  02 
116  -0.4092142E  01 
120  -0.2210315E  02 
124  -0.9601659E  01 
128  -0.1876996E  02 
132  -0.2198848E  02 
136  -0.1463058E  02 
140  -0.2111284E  02 
144  0.0 

148  -0.7800862E  00 
152  0.0 

156  -0.2106975E  02 
160  0.377071 IE  02 
164  0. 1370200E  01 
168  0.4982469E  02 
172  -0. 1550553E  02 
176  0.3560121E  02 

180  -0.5167615E  01 
184  -0.4421077E  02 
188  -0.2669786E  02 
192  0. 1322395E  02 

196  -0.40I8613E  01 
200  0. 1604269E  02 
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TABLE  11 

RELATIVE  IMPORTANCE  VECTOR  FOR  TIME  TO  FAILURE  ESTIMATE 
CONTINUED 


MEAS  REL 

201  0 
205  -0 
209  -0 
213  -0 
217  -0 
221  0 
225  -0 
229  -0 
233  0 

237  0 

241  0 

245  -0 
249  -0 
253  -0 
257  -0 
261  0 
265  -0 
269  -0 
273  -0 
277  0 

201  -0 
285  0 

289  0 

293  0 

297  -0 
301  0 

305  -0 
309  0 

313  -0 
317  -0 
321  0 

325  -0 
329  0. 

333  -0. 
337  -0. 
341  -0. 
345  0, 
349  0, 
353  -0. 
357  0, 

361  0. 

365  0. 
369  0, 

373  0, 
377  -0, 
301  0 

385  -0. 


IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  MEAS  REL 


,  1242579E 

02 

202 

-0.3940600E 

01 

203  -0.2934434E 

01 

204 

-0 

,  1193093E 

02 

206 

-0. 1441004E 

02 

207  -0.7281200E 

01 

208 

0 

U55753E 

02 

210 

-0.2060773E 

02 

211  -0.4121910E 

02 

212 

-0 

, 472856 9E 

01 

214 

-0.7533773E 

01 

215  -0. 1365840E 

02 

216 

0 

4364468E 

01 

210 

0.3549054E 

02 

219  0. 1473612E 

01 

220 

0 

.  23980 77E 

01 

222 

-0.2244514E 

01 

223  0.3102565E 

01 

224 

-0 

4270341E 

02 

226 

0.3147703E 

01 

227  -0. 1344404E 

02 

220 

-0 

,1 06646 7E 

02 

230 

-0 . 1705042E 

02 

231  -0. 1657433E 

01 

232 

0 

.  7881571 E 

00 

234 

0.3224231E 

02 

235  -0. 1016324E 

02 

236 

0 

0951687E 

01 

238 

0.7483651E 

01 

239  0 . 1744193E 

02 

240 

0 

,  3301645E 

02 

242 

0.1 9344 74E 

02 

243  0.5388067E 

00 

244 

-0 

4226152E 

01 

246 

-0.5253316E 

00 

247  -0.4460971E 

00 

248 

-0 

.0731 750E 

02 

250 

-0.7052401E 

01 

251  -0.3562723E 

02 

252 

-0 

5717484E 

01 

254 

-0.2960257E 

01 

255  -0.2131450E 

01 

256 

-0 

5467925E 

01 

258 

0.2457834E 

01 
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MEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE  HEAS  REL  IMPORTANCE  MEAS  REL  IMPORTANCE 


1  0. 1018085E  01 

5  0.1246382E  01 
9  -0. 1505960E  01 
IS  0.9667006E  00 
17  -0.9405985E  00 
21  0 .6723406E  00 

25  -0.5976703E  00 
29  0. 60932 17E  00 

33  O.1129106E  00 

37  0.2920814E  00 

41  0.1149422E  00 

45  -0 .67020 16E-01 
49  -0.6345086E  00 
53  -0.6395231E  00 
57  0.2114109E  00 
61  -0.9517187E-01 
65  0. 1330135E  00 
69  -0.1500145E  01 
73  -0. 1109724E  01 
77  -0 . 32232 36E  00 
01  -0 . 3564538E  00 
65  0.3953172E  00 
69  0.2815247E  00 

93  -0.6711054E  00 
97  0. 1319414E  00 

101  0.3030061E  00 

105  0.2175782E  00 

109  0.4200033E  00 
113  -0.3946757E  00 
117  0.7457741E  00 

121  0 . 2849959E  00 

125  -Q.2070535E  00 
129  -0.3439504E  00 
133  -0.1294952E  01 
137  0.4661023E  00 

141  0.2492840E  01 

145  0.0 

149  0. 1515146E  01 
153  -0.5672472E  00 
157  0 . 1647396E  00 
161  0.1022497E  01 
165  -0.6739447E  00 
169  0.1721919E  01 
173  0.3267555E  00 

177  0.7616421E-01 
161  -0.2168746E  01 
165  -0.6759006E  00 
189  -0. 1246584E  01 
193  0.6194684E  00 
197  -0.2026993E  01 


2  0.0 

6  -0.3721313E  00 
10  -0.8697131E  00 
14  0.7296610E  00 

16  -0.1014250E  01 
22  0.6615616E  00 

26  0.9817330E  00 

30  0.9223944E  00 

34  0.5569226E  00 

36  -0.4687795E  00 
42  -0.2331235E-01 
46  0.3703020E  00 

50  -0.6399178E  00 
54  -0.3462794E  00 
56  0.7903212E-01 

62  0.4733251E  00 

66  0.6072890E  00 

70  -0.1745461E  01 
74  -0.9440714E-01 
78  0.7660243E-01 

62  0.5625586E-01 

86  0.7165062E  00 
90  0.6669376E-01 

94  0.6429899E  00 

98  0.2436669E  00 

102  -0.4916456E  00 
106  0 . 2946643E  00 

110  -0 .2641 466 E- 01 
114  -0.6624296E  00 
116  0.5261017E  00 
122  -0.5936578E  00 
126  -0.1635666E  00 
130  -0. 157536 9E  00 
134  0.U76556E  01 

136  >0 . 3668765E  00 
142  -Q.2071362E  01 
146  0.0 

150  0.5764603E  00 

154  -0.7795215E  00 
156  0. 1612967E  01 
162  -0.6450409E  00 
166  -0.1982259E  01 
170  0.2096299E  00 

174  0. 1266099E  01 

1,4  -0.2881049E  00 
162  0. 1713390E  01 
166  -0.2757357E  00 
190  0. 1525523E  01 

194  -0. 1165631E  01 
198  -0.247151QE  00 


3 

-0. 1069779E 

01 
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-0.3763975E 

00 

11 

-0. 7058561E-02 

15 

-0.2096103E 

00 
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-0.9794769E 

00 

23 

0.6659415E 

00 

27 

0.1365042E 

01 

31 

0.5164803E 

00 

35 

0.6395934E 

00 

39 

0.2259300E 

00 

43 

0.6949965E-01 

47 

0.5503922E 

00 

51 
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00 

55 
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00 
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00 
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00 
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00 
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01 
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-C.2193064E 

01 
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0.4323676E 

00 
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0.6016109E 

00 
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0. 1294640E 

01 

4  0.0 

6  -0.2357730E  00 
12  0. 1003672E  01 

16  -0.3667985E  00 
20  0.5846576E  00 

24  -0.3767015E-01 
26  0.2816179E  00 

32  -0 . 7633207E-01 
36  0. 1750003E  00 

40  0. 1070677E  01 

44  0.1 0096 98E  01 

46  0. 1 194916E  01 

52  0.2453933E  00 

56  0. 1097836E  01 

60  -0.1708776E  00 
64  0. 1074327E  01 

68  0. 1642430E  00 
72  -0. 1504921E  00 
76  0.2560797E  00 

60  -0.4704635E  00 
64  0.1417637E  00 
86  0.6201147E  00 
92  -0.2671614E-01 
96  -0. 390236 9E  00 
100  -0.2259531E-01 
104  0.7482947E  00 

108  0.4041923E  00 

112  0.7595253E-01 
116  -0.1448764E  01 
120  -0. 1066514E  01 
124  0.7033674E  00 

126  -0.6479465E  00 
132  -0.7496416E  00 
136  -0. 204156 7E  01 
140  -0.3301761E  01 
144  O'  0 

140  -0. 17066 75E  01 
152  0.0 

156  -0.9545544E  00 
160  -0 . 3573328E -0 1 
164  -0.2578800E  00 
166  0.4510049E  00 

172  -0.9505637E  00 
176  -0.5093591E  00 
160  -0.3009304E  01 
164  -0.5263692E  00 
166  -0.2266396E  01 
192  0 . 2796764E  00 

196  -0.6635628E  00 
200  0.6394655E-01 
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201 

-0.2081120E 

00 

202 

-0.3256350E  00 

205 

-0.3259713E 

01 

206 

-0.2023207E-01 

209 

-0.2132229E 

01 

210 

0.7317746E  00 

213 

-0 . 1236606E 

01 

214 

0.4934639E  00 

217 

-0.4070061E 

00 

210 

-0. 469290 3E  00 

221 

0.2001110E 

00 

222 

0.4256492E  00 

225 

-0. 689246 7E 

00 

226 

0. 1613944E  00 

229 

0.2312072E 

01 

230 

0. 1740284E  01 

233 

0.5046159E 

00 

234 

0.5874126E  00 

237 

-0. 18590 10E 

01 

238 

0.6441610E  00 

241 

0. 1S05892E 

01 

242 

0.3613245E-03 

245 

-0.2400309E 

00 

246 

0.3385388E  01 

249 

-0.34U261E 

00 

250 

-0.1107679E  01 

253 

0.8463031E- 

■01 

254 

0.7749497E  00 

257 

-0.3727073E 

00 

258 

0.2928561E-01 

261 

-0.3817562E 

00 

262 

-0.6634004E-02 

265 

-0.2861098E 

00 

266 

-0.2212636E  00 

269 

0.2175265E 

01 

270 

0.4804091E  00 

273 

0. 1590540E 

01 

274 

-0.36279S8E  00 

277 

-0.2528992E 

00 

278 

0.6981200E-01 

201 

0. 756836 7E 

00 

262 

0.0 

205 

0.0 

286 

0.5619595E-01 

209 

0.0 

290 

0.0 

293 

0.0 

294 

0.5619595E-01 

297 

-0.4516575E 

00 

298 

0.7069039E  00 

301 

0.0 

302 

0.2180402E  01 

305 

0.2995596E- 

-01 

306 

0.0 

309 

0.0 

310 

0.0 

313 

0.2540762E 

01 

314 

0.3256267E-01 

317 

0.3120646E 

00 

318 

-0 . 7007414E-0 1 

321 

0.0 

322 

0.0 

325 

-0.1132333E 

00 

326 

0.0 

329 

0.0 

330 

0.0 

333 

0.4789465E-01 

334 

0.3256287E-01 

337 

-0. 1348047E 

00 

338 

0.0 

341 

0. 1521866E 

01 

342 

0.0 

345 

0.0 

346 

0.0 

349 

0.0 

350 

0.0 

353 

0.4769465E-01 

354 

-0.7007414E-01 

357 

0.0 

356 

0.0 

361 

0.0 

362 

0.0 

365 

-0. 9761601 E 

00 

366 

0.2826762E  01 

369 

-0. 1481496E 

01 

370 

0.0 

373 

0.0 

374 

0.0 

377 

0.4526872E-01 

376 

0.1097348E  01 

361 

0.0 

382 

0.0 

365 

-0.1486161E 

01 

203 

-0. 1368968E 

00 

204 

-0.3126664E 

01 

207 

0.7548226E 

00 

206 

0.7526026E 

00 

211 

-0.6663212E 

00 

212 

-0. 1465303E 

01 

215 

0.7470947E 

00 

216 

0.3994610E 

00 

219 

-0.5322373E 

00 

220 

0. 1331966E 

01 

223 

0. 4458765 E 

00 

224 

-0. 1708074E 

00 

227 

-0.1 2996 76E 

01 

228 

0.8178920E 

00 

231 

-0.1I47779E 

01 

232 

0.2889165E 

00 

235 

0. 1361331E 

01 

236 

0.2167643E 

01 

239 

-0. 1064203E 

01 

240 

-0.3272750E 

00 

243 

-0.7066644E 

00 

244 

0.3046526E 

01 

247 

0.2494578E 

01 

246 

-0.3063066E 

00 

251 

-0. 1185490E 

01 

252 

-0.1612706E 

00 

255 

0.6065061E 

00 

256 

-0.3986626E 

00 

259 

0.6476235E 

00 

260 

-0.5436526E 

00 

263 

0.2172058E 

00 

264 

-0.9072200E 

00 

267 

-0.4516054E 

00 

266 

-0. 1661353E 

00 

271 

-0.2253542E 

01 

272 

0 . 3763053E 

00 

275 

0.1718975E 

01 

276 

0. 57966 74E 

00 

279 

0.0 

260 

0.0 

263 

0.2180402E 

01 

264 

0.2032266E 

01 

267 

-0.2712603E 

00 

286 

0.5699629E-01 

291 

0.4461670E 

00 

292 

0.0 

295 

0.0 

296 

0.0 

299 

0.0 

300 

0.0 

303 

-0. 5387750 E 

00 

304 

0.0 

307 

0.0 

306 

0.0 

311 

0.5619595E- 

-01 

312 

0 . 7769760E 

00 

315 

0.0 

316 

0.0 

319 

0.0 

320 

0.0 

323 

-0.3129936E 

00 

324 

0.0 

327 

0.0 

328 

0.0 

331 

0.0 

332 

0.0 

335 

0.0 

336 

0.0 

339 

-0.1467619E 

01 

340 

-0.7007414E-01 

343 

0.0 

344 

347 

0.0 

346 

0.0 

351 

0.6308983E 

00 

352 

0.0 

355 

0.6076094E 

00 

356 

0.0 

359 

0.0 

360 

0.0 

363 

0.0 

364 

0.0 

367 

0.4526872E-01 

366 

0.3277752E 

01 

371 

0.0 

372 

0.0 

375 

0 . 3765250E 

00 

376 

0.2148601E 

01 

379 

0. 3401626E-01 

360 

0.0 

363 

0.0 

384 

0.0 
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expected  as  new  oil  adjusts  to  an  equilibrium.  A  larger 
and  more  complete  sample  of  engines  would  eliminate  this 
discriminant. 

A  second  feature  of  importance  is  that  the  NOAP 
components  most  important  to  the  separation  of  manually 
vs  automatically  recorded  data  are  not  important  to  any 
other  classification  studied.  This  suggests  that  these 
differences  will  have  a  negligible  effect  on  the  present 
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This  attachment  will  present  the  detail  information 
which  defines  the  ADAPT  approach* to  empirical  data  analysis. 

This  approach  is  based  on  the  concept  that  empirical  data 
analysis  should  be  preceded  by  transforming  the  data  from 
the  original  data  space  to  a  more  efficient  analysis  space. 

This  more  efficient  analysis  space  is  defined  as  that  space 
which  requires  the  least  number  of  numbers  to  represent  a 
given  amount  of  information  in  the  original  data  set.  It 
can  be  shown  that  this  space  is  simply  the  eigenvector  space 
and  the  transformation  required  is  the  eigenvector  or  the 
Karhunen-Loeve  transformation. 

The  personnel  who  are  now  the  senior  technical  staff  of 
the  ADAPT  Service  Corporation  each  have  a  decades  experience 
with  analysis  in  the  eigenvector  space.  This  has  led  to  the 
development  of  aunique  set  of  computer  programs  both  to  perform 
the  transformation  to  the  eigenvector  space  and  to  perform  the 
analysis  in  this  space. 

The  ADAPT  programs  have  many  outputs  which  are  considerably 
different  from  those  which  are  obtained  from  classical  approaches 
to  empirical  or  statistical  analysis.  This  attachment  will 
attempt  to  present  a  brief  description  of  these  outputs  and  how 
they  may  be  used  to  improve  empirical  data  analysis.  In  the 
following  paragraphs,  we  will  summarize  each  of  the  capabilities 
and  outputs  of  the  ADAPT  analysis  procedure. 

ADAPT  OPTIMAL  REPRESENTATION 

The  major  difference  between  the  ADAPT  approach  to  empirical 
analysis  and  the  classical  approach  to  empirical  analysis  is  the 
derivation  and  use  of  the  ADAPT  optimal  representation  to  simplify 
and  improve  all  subsequent  empirical  analysis  of  the  data.  The 
ADAPT  optimal  representation  is  known  in  the  literature  under 
the  names  of:  1)  principal  component  analysis,  2)  Karhunen-Loeve 
expansion,  3)  eigenfunction  expansion  and  4)  optimum  empirical 
orthogonal  functions.  The  ADAPT  Service  Corporation  has  developed 
a  unique  approach  to  obtaining  this  transformation  which  overcomes 
the  difficulties  associated  with  the  iterative  techniques  discussed 
in  the  literature  and  available  in  most  "statistical' packages" . 

The  importance  of  this  unique  approach  to  deriving  eigenvectors 
is  discussed  in  the  ADAPT  write-up  titled  "significance  of  ADAPT 
Approach  to  Deriving  Eigenvectors"  included  as  Appendix  2B. 

It  is  useful  to  review  some  of  the  basic  concepts 
associated  with  the  ADAPT  optimal  representation.  The 
first  point  which  must  be  established  is  the  meaning  of 
optimal.  For  the  ADAPT  application,  optimal  is  defined 
as  that  representation  which  requires  the  least  number  of 
numbers  to  represent  a  given  amount  of  information  or  varia¬ 
tion.  Thus,  by  definition,  the  ADAPT  optimal  representation 

t 


is  the  most  efficient  orthogonal  coordinate  system  for 
representing  the  learning  data.  -For  further  discussion  of 
this  transforma  tior^  and  its  use  see  References  1-3. 

After  the  optimal  representation  has  been  obtained, 
the  learning  data  is  transformed  to  the  optimal  space 
and  the  analysis  is  performed  in  the  optimal  space.  The 
optimal  space  may  be  viewed  either  geometrically  as  a 
new  coordinate  system  for  describing  the  learning  data  of 
functionally  as  a  system  of  empirical  orthogonal  functions 
(EDF)  to  be  used  to  construct  a  generalized  Fourier  series 

representation  of  the  learning  data.  In  the  first  case, 
the  analysis  is  performed  on  the  coefficients  of  each 
of  the  data  vectors  in  the  new  space.  In  the  latter  case, 
the  analysis  is  performed  on  the  coefficients  of  the 
generalized  Fourier  series  expansion  of  each  of  the 
data  histories.  The  numerical  value  of  the  coefficients 
is  identical  regardless  of  whether  the  procedure  is 
visualized  geometrically  or  functionally.  Thus,  the 
majo.-  output  of  the  first  step  of  any  ADAPT  analysis  is 
the  transformation  matrix  to  transform  the  data  from  the 
original  data  space  in  which  the  data  vectors  are  defined 
to  the  new  optimal  ADAPT  analysis  space. 

To  visuelize  the  role  of  the  ADAPT  representation, 
consider  the  transformation  matrix  Hn  between  the  original 
data  space  containing  observation  or  data  vectors  "SV^n* 
and  the  optimal  analysis  space  containing  the  transformed 
data  vectors  “Y]jr".  Figure  1  presents  a  block  diagram  of 
the  ADAPT  process  illustrating  this  role  of  the  optimal  space. 

The  transformation  matrix,  Hn,  is  an  orthogonal  matrix,  the 
inverse  of  this  transformation  is  equal  to  its  transpose. 

Thus,  one  has  rules  for  transforming  the  data  to  the  optimal 
space  and  the  results  of  the  analysis  from  the  optimal  space 
back  to  the  original  space.  The  dimensionality  of  the  original 
space  which  using  the  notation  of  Figure  1  is  K  by  n  will  be 
reduced  to  K  by  r  where  K  is  the  number  of  cases  and  n  is  the 
number  of  dimensions  or  number  of  numbers  required  to  describe 
each  case.  In  general,  for  large  data  sets,  r  is  at  least  an 
order  of  magnitude  less  than  n.  For  data  vectors  less  than 
the  order  of  one  hundred,  r  may  only  be  a  factor  of  2  to  10 
less  than  n.  The  data  in  the  original  data  space  is  designated 
by  the  symbol  vkn*  In  the  optimal  space,  his  data  is  represented 
by  the  coefficients  Y^j-.  Where  K  in  both  cases  designates  the 
case  and  n  and  r  designate  the  components  of  the  data  vector 
in  each  of  the  spaces,  respectively.  One  may  transform  the 
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data  either  from  the  original  data  space  to  the  optimal 
space  or  visa  versa  by  using  the  transformation  matrix 
as  indicated  by  tha.  arrows  on  Figure  1.  Linear  algorithms 
may  also  be  transformed  between  the  data  space  and  the 
optimal  space  by  use  of  the  H  matrix. 

The  ADAPT  characteristics  which  in  addition  to  the  classi¬ 
cal  statistical  summary  parameters  would  be  of  interest  include 
the  ADAPT  optimum  function,  the  information  energy  plot,  the 
ADAPT  scatter  plot,  the  ADAPT  relative  importance  vector, 
performance  map,  independent  eigenscreening  and  the  empirical 
validity  criteria.  The  following  paragraphs  will  present 
a  brief  description  of  each  of  these  and  some  of  the  ADAPT 
preprocessing  concepts. 


Optimum  Function 

Referring  to  the  preceding  descrption  of  the  ADAPT 
process,  the  ADAPT  optimum  function  is  numerically  the 
corre.;,  ponding  column  of  the  H  matrix.  Since  this  vector 
is  de.'  :ibed  by  N  components,  it  has  the  appearance  of  a 
data  \  or  that  shows  the  importance  of  each  of  the 
origi.'  omponents  of  the  data  vectors  to  the  constru  ‘-.ion 
of  th<  imal  space.  Plots  of  this  function  provide  a 

physi.  uterpretation  for  the  components  of  the  optimal 

space  also  an  indication  of  which  of  the  original  data 

vector  ^.-mpcnents  are  conveying  similar  information.  r.his 
may  be  viewed  as  an  analysis  of  variation  of  the  data 
but  it  should  not  be  confused  with  the  classical  analysis 
of  variation  which  is  normally  associated  with  the  outputs 
of  regression  analysis.  These  classical  analyses  of 
variation  generally  describe  how  much  of  the  variation 
observed  in  the  dependent  variable  can  be  explained  by 
the  independent  variable.  The  ADAPT  optimal  functions  on 
the  other  hand,  are  simply  an  analysis  of  variation  of  the 
independent  variable  without  considering  the  dependent 
variable  at  all.  It  seeks  to  answer  the  question  which 
independent  variables  express  the  greatest  amount  of 
variation  and  which  independent  variables  convey  similar 
information. 

Information  Energy  Plot 

The  eigenvalues  associated  with  each  of  the  optimal 
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functions' defines  the  amount  of  variation  in  the 
learning  data  set  which  is  explained  by  that  optimal 
function.  Since^all  information  must  be  conveyed  by 
variation  in  the  data,  this  variation  is  analogous 
to  an  "information  energy".  One  of  the  standard  ADAPT 
outputs  which  will  be  provided  -for  each  of  the  bases 
developed  (i.e.  transformation  matrices)  in  this  study 
is  a  plot  of  this  information  energy  or  eigenvalue 
as  a  function  of  a  number  of  dimensions  used.  Examina¬ 
tion  of  this  information  energy  curve  allows  one  to 
determine  the  dimensionality  at  which  the  information 
has  the  character  of  noise.  One  can- also  observe  the 
change  of  character  of  the  information  represented  as 
a  function  of  dimensionality.  It  is  often  possible  to 
detect  the  point  at  which  the  eigenvectors  are  primarily 
correcting  for  anomolous  cases!  Thus,  the  information 
energy  is  one  of  several  important  'tools  in  selecting 
the  dimensionality  for  the  analysis.  Some  of  the  subtle 
aspects  associated  with  analysis  of  the  information 
energy  are  discussed  in  references  4-6.  Ref-4  is  a  funda¬ 
mental  paper,  the  results  are  often  misused.  Ref-6  discusse 
this  in  more  detail. 


Scatter  Plot 

The  ADAPT  scatter  plot  is  the  projection  of  the 
data  vectors  under  consideration  on  two  dimensions  of 
the  optimum  space.  In  general,  one  projects  on  the 
first  two  dimensions  on  the  optimal  space  since  these 
two  dimensions  provide  the  best  representation  of  the 
information  contained  in  the  data.  This  is  identical 
to  making  a  scatter  plot  of  the  Y-^  versus  Y2jc  coeffic¬ 
ients.  Note,  that  equation  2  on  Figure  1  can  be  inter¬ 
preted  as  the  generalized  Fourier  series  expansion  of 
data  history  SV^  in  terms  of  the  orthogonal  functions 
defined  by  the  H  matrix.  Thus,  a  data  history,  SVpn 
having  a  first  coefficient  of  1,  (Ylp  =  1) ,  on  the 
scatter  plot  and  a  second  coefficient  of  -1,  (Y2p  =  -1) , 
would  have  a  two  term  generalized  Fourier  series 
representation  equal  to  the  difference  between  the  first 
and  second  optimal  functions  (SVp  -  H-^-  H2  ...).  The 
significant  achievement  of  the  scatter  plot  of  the  first 
two  optimal  coefficients  of  each  of  the  data  vectors 
is  that  it  presents  the  best  possible  two  dimensional 
representation  of  the  entire  data  set.  Each  point  on 
the  scatter  plot  represents  an  entire  history  made  up  of 
N  points. 


Algorithm  and  Relative  Importance  Vectors 

-  *» 

The  derivation  of  a  linear  classification  algorithm 
may  be  looked  upon  as  the  search  for  the  line  or  vector 
with  the  property  that  the  numerical  value  of  a  data 
vector !s  projection  on  this  line  is  a  good  detection 
statistic.  The  ADAPT  algorithm  vector  is  a  plot  of 
the  components  of  the  projection  of  this  vector  in 
the  original  data  space.  Since  the  dot  product  of  this 
vector  wit1  the  data  vector  determines  the  detection 
statistic,  the  magnitude  of  each  of  these  components 
provides  a  measure  of  the  importance  of  each  component 
to  the  algorithm  being  evaluated.  In  the  ADAPT 
programs,  the  algorithm  vector  is  derived  in  the  optimal 
space.  Thus  in  data  space  this  vector  is  the  product  of 
the  vector  defined  in  the  optimal  space,  Ar  times  the 
transformation  matrix  Hnr • 

The  importance  of  any  variable  to  an  algorithm  is 
the  product  of  two  values:  1)  the  value  of  the  algorithm 
associated  with  that  value  and  2)  the  amount  of  variation 
associated  with  the  variable.  For  example,  a  given 
variable  makes  no  contribution  to  an  algorithm  if  the 
algorithm  value  is  zero  or  if  it  has  the  same  value  for 
all  observations.  Thus,  we  define  the  relative  importance 
vector  as  a  vector  in  data  space  where  each  component 
is  the  product  of  the  algorithm  value  and  the  variance 
of  the  variable  associated  with  that  component.  It 
follows  from  the  mechanism  of  the  dot  product  operation 
that  it  is  the  absolute  value  of  the  relative  importance 
(or  algorithm)  vector  which  is  significant.  For  example, 
considering  the  algorithm  vector,  if  one  variable  has  a 
value  of  minus  .5  and  another  variable  a  value  of  plus  0.1 
a  change  in  the  indexing  variable  having  the  value  of  minus 
0.5  in  the  algorithm  vector  has  five  times  the  effect  on 
the  answer  or  detection  statistic  as  the  same  change  in 
the  indexing  variable  having  a  value  of  plus  0.1. 


Performance  flap 


The  performance  map  is  a  plot  of  the  dimensionality 
used  for  the  analysis  versus  the  performance  of  the  algorithm 
developed.  It  provides  an  empirical  non-parometric  tool 
to  determine  whether  there  was  sufficient  learning  cases 
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to  provide  a  physically  meaningful  algorithm.  It  also 
provides  a  tool  for  estimating  the  gains  possible  by 
increasing  the  amount  of  training  data.  The  task  accomplished 
is  analogous  to  the  problem  of  fitting  a  third  order  polynomial 
to  independent  test  data.  One  can  always  fit  a  third  order 
polynominal  to  three  numbers,  and  there  is  no  implied  physical 
significance  to  the  fact  that  there  is  a  good  fit.  This  is 
often  referred  to  as  an  overdetermined  problem.  On  the  other 
hand,  if  one  has  a  "large"  number  of  independent  samples  say 
one  hundred  samples  and  one  fits  a  curve  to  this  larger  set 
of  samples,  one  may  conclude  that  those  hundred  samples  can 
be  approximated  by  a  third  order  polynomial  expression  over 
the  range  of  the  available  experimental  data. 


The  question  is,  what  is  "large"?  The  same  phenomena 
occurs  for  all  empirical  analysis.  If  the  number  of 
learning  cases  equals  the  number  of  dimensions,  most 
empirical  algorithms  will  fit  the  learning  data  exactly, 
however,  once  agairt  there  is  no  physics  implied  in  this 
fit.  As  one  increases  the  number  of  learning  cases  beyond 
this  point,  if  one  continues  to  achieve  good  fits  of  the 
data  with  the  empirical  algorithm,  the  probability  that  the 
fit  is  based  on  physics  increases.  Eventually  when  the 
ratio  of  learning  cases  to  number  of  dimensions  used  is 
"sufficiently  large",  one  not  only  can  assume  that  the 
relationship  is  based  on  physics  but  that  the  performance, 
which  is  obtained  on  the  learning  data  may  safely  be 
extrapolated  to  future  independent  test  samples.  The 
ADAPT  performance  map. can  be  used  to  define  "sufficient  large". 


After  introduction  of  the  independent  eigenscreening 
concept  into  the  ADAPT  linear  classification  and  regression 
programs,  the  performance  map  was  no  longer  required 
to  determine  if  the  overdetermined  situation 
had  been  obtained.  However,  the  performance  maps  are  now 
easier  to  use  and  still  determine  if  additional  training 
data  should  be  used.  They  now  provide  plots  of  both  the 
biased  and  unbiased  performance  as  a  function  of  ratio 
of  number  cases  to  dimensionality.  When  both  the  biased 
and  unbiased  performances  are  similar,  the  number  of 
training  cases  are  adequate  for  that  algorithm. 

Empirical  Validity  Criteria 

The  ADAPT  approach  of  preceding  the  empirical  analysis 
with  an  optimal  representation  also  provides  a  mechanism 


for  performing  a  necessary  but  not  sufficient  test  to 
determine  whethefc  an  empirical  algorithm  is  applicable 
to  a  new  independent  test  sample.  This  empirical  validity 
criteria  consists  of  obtaining  the  ratio,  (Q)  of  the  length  of 
the  data  vector  for  the  new  independent  test  case  in  the 
optimal  space  to  its  length  in  the  original  data  space. 

If  this  ratio  is  significantly  less  than  the  corresponding 
ratio  for  the  average  or  typical  learning  data  used  to 
derive  the  algorithms,  the  independent  test  case  has  been 
obtained  from  a  sample  which  is  significantly  different 
from  the  learning  data.  Thus,  empirical  analysis  of  the 
test  case  based  on  algorithms  derived  from  that  learning 
set  can  not  be  justified.  Experience  with  this  validity 
criteria  in  many  different  problems,  has  shown  it  to  be 
very  effective  in  providing  apriori  estimate  of  whether 
an  algorithm  is  applicable  to  a  particular  test  case. 

This  procedure  has  been  part  of  the  ADAPT  family  of  computer 
programs  and  was  first  described  in  the  literature  in  Ref— 7. 

Grpup-Out  Independent  Testinci 


The  ADAPT  regression  and  classification  algorithm 
development  programs  include  a  capability  to  obtain 

independent  (i.e.  unbiased)  test  results  with  a  minimum 
increase  in  the  required  number  of  cases.  This  is 
achieved  through  the  group-out  testing  procedure.  The 
procedure  is  to  consider  the  original  training  set  of 
data  as  made  up  of  a  relatively  large  number  of  small 
groups  of  cases.  Note,  that  the  group  may  be  as  small 
as  one  case.  If  we  have  a  set  of  M  cases  available 
for  the  study  and  we  use  groups  of  N  cases  each,  the 
procedure  is  to  remove  the  first  group  of  N  cases  giving 
a  training  set  of  M  minus  N  cases  and  an  independent  test 
set  of  N  cases.  The  algorithms  are  derived  on  the 
training  set  and  tested  against  the  N  cases  in  the  first 
group.  When  this  is  completed,  the  N  cases  in  the  first 
group  are  returned  to  the  training  set  and  a  second 
group  of  N  cases  is  removed  and  the  procedure  repeated. 

If  this  procedure  is  followed,  one  finds  that  they  have 
derived  a  total  of  M  divided  by  N  algorithms  each  having 
M  minus  N  training  cases  and  has  tested  the  total  of  all 
of  these  algorithms  against  M  independent  test  cases. 
Thus,  the  net  effect  of  this  procedure  is  to  effectively 
provide  M  minus  N  training  cases  and  M  independent  test 
cases  from  a  set  of  M  cases. 


It  should  be  noted,  that  the  procedure  originally  reported 
in  the  literature  in  Ref-8  is  based  on  the  capability  to  obtain 
an  inverse  with  one  case  omitted  from  the  covariance  matrix.  In 
the  ADAPT  programs,  our  ability  to  use  this  procedure  is 
due  to  the  efficiency  of  the  analysis  in  the  ADAPT  space. 
Although  we  also  have  some  programs  which  make  use  of 
the  procedure  outlined  in  the  literature  combined  with 
the  efficiency  of  the  ADAPT  analysis  space  which  provides 
an  extremely  economical  way  of  performing  one-out  testing. 

We  have  compared  the  performance  obtained  with  the  group- 
out  testing  with  classical  independent  testing  and  have 
found  with  random  selection  of  groups,  stable  sets  of 
algorithms  produce  identical  results  as  independent  tests 
when  training  and  test  samples  are  drawn  from  homogeneous 
data.  With  conservative  selection  of  groups  the  group-out 
testing  is  a  more  severe  test. 


Classical  screening  regression  has  been  avoided  in 
the  development  of  the  ADAPT  computer  programs  for  two 
reasons.  These  reasons  are:  1)  classical  screening 
regression  makes  the  screening  decision  based  on  the 
performance  established  from  training  data.  Comparative 
analyses  between  use  of  independent  test  data  and 
training  data  performed  by  the  ADAPT  Service  Corporation 
ha’ 3  shown  that  the  training  data  does  not  provide  a 
reasonable  basi3  for  screening  of  the  variables  and  2) 
classical  screening  is  performed  on  a  set  of  independent 
variables  which  are  not  orthogonal  and  thus  considerable 
effort  is  required  ascertain  whether  a  variable  is 
retained  because  it  is  significant  or  because  it  is 
repeating  information  which  has  already  been  obtained  in 
a  different  variable. 

The  ADAPT  eigenscreening  approach  is  similar  to  the 
classical  screening  regression  except  that  the  screening 
is  performed  in  eigenvector  space  and  performance  is 
established  based  on  the  group-out  testing  procedure 
and  thus  is  based  on  independent  and  unbiased  test  results. 
Since  the  screening  is  performed  in  the  eigenvector  space 
instead  of  the  data  space,  the  variables  being  screened 
are  orthogonal  and  one  need  not  be  concerned  with  the 
linear  dependence  between  the  screened  variables. 


The  screening  process  is  significantly  improved 
because:  1)  th^ unbiased  test  provides  a  higher  confidence 
performance  estimate  than  dependent  testing  and  2)  the 
group-out  testing  allows  the  evaluation  of  the  stability 
of  each  term  in  the  algorithm  as  well  as  the  algorithm 
performance.  The  evaluation  of  the  stability  is  especially 
important  when  the  number  of  cases  is  limited,  since  the 
"overdetermined"  solution,  which  must  be  avoided,  is  very 
unstable.  If  the  performance  of  the  different  algorithms 
developed  in  the  group-out  testing  is  unstable,  one  can  be 
certain  that  there  are  insufficient  training  cases.  If 
any  term  in  the  algorithm  is  unstable,  this  term  probably 
saould  have  been  rejected. 

• 

These  improvements  in  the  screening  have  resulted  in 
significant  additional  capabilities  in  performing  regression 
analysis.  The  ADAPT  Service  Corporation  has  also  applied 
these  procedures  to  pattern  recognition  techniques  and  has 
computer  programs  which  provide  these  same  advantages  to 
the  development  of  classifxcation  algorithms.  Further  dis¬ 
cussion  and  examples  illustrating  the  ADAPT  ei genscreening  are 
given  in  the  ADAPT  write-up  "Illustration  of  ADAPT  Independent 
Eigenscreening  Technique",  included  as  Appendix  C3D. 


Preprocessing 

Ar.  extremely  important  factor  in  obtaining  good 
empirical  results  is  to  preprocess  the  data  such  that  the 
information  is  presented  in  a  useful  manner.  The  ADAPT 
family  of  computer  programs  include  the  capability  to  provide 
most  of  the  classical  preprocessing, such  as  normalizations, 
adjusting  the  data  according  to  some  prescribed  function, 
taking  Fourier  or  cepstrum  transforms  of  the  data.  The 
ADAPT  computer  programs  also  include  specialized  preprocess¬ 
ing  which  has  been  developed  based  on  requirements 
established  as  a  result  of  the  work  performed  in  the  past. 
These  include  such  techniques  as  equalization,  thresholding 
and  a  unique  capability  for  objectively  deriving  folding 
procedures  to  overcome  non-linearities  and  non-roono tonic 
relations  between  the  predictant  and  the  predictor  variable. 

The  last  preprocessing  performed  before  processing  the 
data  through  the  ADAPT  eigenvector  derivation  programs 
or  transforming  the  data  to  the. eigen 'pace  is  to  reduce 
the  data  to  zero  mean  by  subtracting  ,e  average  of  all 
the  training  cases  from  each  data  vfc  ,»r.  The  zero  mean 
data  offers  a  great  many  numerical  advantages  and  is  used 
in  almost  all  ADAPT  studies. 


V 

In  all  studies  where  different  types  of  variables 
(eg.  a  data  vector  composed  of  temperature  measurements 
and  pressure  measurements)  and  many  cases  where  the 
variables  are  the  same  but  their  magnitudes  may  mask 
the  variation,  the  reduction  to  zero  mean  is  preceded 
by  " equalization"  of  the  data  vector.  This  is  a  process 
by  which  the  value  of  each  variable  is  forced  to  lie  between 
1.0  and  2.0.  This  is  accomplished  by  transforming  the 
original  variable  Vfc(x)  to  a  new  variable  Vj^x)  using: 

Vk(x)  -  VMIN(x) 

Vfc(x)  s  i  +  VMAX(x)  -  VMIN(x)  (1) 

where  V^fx)  s  Value  of  observation  k  associated  with  index  x 

X  s  A  range  of  one  or  more  indexing  variables 

VMAX  a  Max  value  over  all  training  data  associated 
with  index  x 

VMIN  a  Min  value  over  all  training  data  associated 
with  index  x 
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APPENDIX  3  -  SIGNIFICANCE  OF  ADAPT  APPROACH  TO  DERIVING  EIGENVECTORS 

♦ 

K  • 

INTRODUCTION 

The  ADAPT  approach  to  empirical  data  analysis  is-  that 
empirical  data  analysis  such  as  pattern  recognition  or  regression 
should  be  preceded  by  transforming  all  of  the  data  into  the 
appropriate  eigenvector  space  for  analysis.  This  provides  an 
optimum  (in  the  Karhunen-Loeve  sense)  space  in  which  to  perform 
the  analysis  and  significantly  decreases  the  cost  and  increases 
what  can  be  learned  from  any  subsequent  analysis.  This  approach 
translates  most  of  the  major  numerical  analysis  problems  into 
the  first  step  (i.e.  finding  the  eigenvectors  of  the  covariance 
matrix  derived  from  the  original  data  vectors) .  Thus,  the 
efficient  and  correct  derivation  of  the  eigenvectors  associated 
with  a  covariance  matrix  is  one  of  the  most  important  aspects 
of  the  ADAPT  approach  to  empirical  analysis. 

The  ADAPT  Service  Corporation  uses^  a  unique  approach  to 
the  derivation  of  these  eigenvectors  which  provides  both  a 
greater  efficiency  with  respect  to  computer  running  time  and 
core  size  and  also  eliminates  the  problems  resulting  from  noisy 
and/or  ill-conditioned  real  data  sets.  These  noise  and  data 
conditioning  problems  are  very  similar  to  the  problems  which 
lead  to  singular  matrices  when  analyzing  data  in  the  original 
data  space.  Although  these  problems  do  not  cause  a  failure 
to  obtain' an  answer  with  conventional  eigenvector  techniques 
such  as  those  included  in  the  IBM  scientific  sub-routine 
package  they  often  lead  to  meaningless  outputs  from  these 
techniques  and  unnecessarily  large  requirements  for  core  si^e 
and  running  time.  This  appendix  will  review  these  difficulties 
and  outline  the  advantages  of  circumventing  these  difficulties 
prior  to  entering  the  procedures  for  deriving  eigenvalues  and 
eigenvectors. 

PITFALLS  OF  CONVENTIONAL  EIGENVECTOR  DERIVATIONS 

Since  we  are  dealing  with  the  task  of  finding  the  eigenvectors 
of  the  covariance  matrix,  we  may  limit  the  discussion  to  real 
symmetric  matrices.  Modern  techniques  (i.e.  the  Jacobi  technique 
which  is  used  in  the  IBM  scientific  sub-routine  package  or  the 
Givens-House-holder  technique  described  in  Reference  1  and  used 
in  many  commercially  available  statistical  packages  are  based 
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on  iterative  techniques  which  usually  proceed  from  some 
initial  guess  for  the  eigenvalues  and  an  apriori  specified 
accuracy.  With  the  judicial  use  of  overflow  and  underflow 
protections  in  the  programming  of  these  techniques,  one 
obtains  a  set  of  numbers  and  vectors  which  look  like  eigen¬ 
values  and  eigenvectors.  In  many  ways,  this  is  unfortunate 
because  unlike  the  situation  with  matrix  inversion  where 
ill-conditioned  input  data  leads  to  the  impossibility  of 
obtaining  an  answer,  ill-conditioned  data  leads  to  a  partially 
incorrect  answer  with  these  eigenvector  techniques.  These 
incorrect  outputs  are  responsible  for  many  of  the  miscon¬ 
ceptions  concerning  eigenvector  analysis  that  are  often 
heard  and  occasionally  even  appear  in  the  literature  regarding 
the  use  of  eigenvectors  as  an  analysis  tool.  The  most  common 
of  these  misconceptions  are: 

1)  the  instability  of  eigenvectors  (l.e.  cases  where 
eigenvectors  corresponding  to  relatively  large  eigenvalues 
are  supposedly  unstable  as  one  changes  the  data  slightly), 

2)  the  statement  that  the  derivation  of  eigenvectors 

for  large  real  data  vectors  is  nearly  computionally  impossible 
(2b:  Reference  2,  Page  31)  and, 

3)  only  the  first  'few  dominant  eigenvectors  can  have 
physical  meaning  (the  ADAPT  Service  Corporation  has  found 

and  verified  physically  meaningful  information  in  eigenvectors 
explaining  considerably  less  than  1%  of-  the  variation) . 

In  the  following  paragraphs,  we  will  discuss  two  problems 
which  may  lead  to  such  false  conclusions,  these  are: 

1)  insufficient  independent  observations  and, 

2)  noise. 

The  inpact  of  insufficient  observations  can  be  seen  most 
clearly  by  considering  a  simple  case.  Suppose  for  example 
one  had  three  observations  (i.e.  cases)  of  some  phenomena 
where  each  observation  consisted  of  five  independent  measure¬ 
ments  associated  with  the  phenomena  being  observed.  This  will 
provide  a  data  matrix  consisting  of  three  vectors  of  five 
components  each.  Clearly,  if  one  attempted  to  run  a  five 
dimensional  regression  or  a  discriminate  analysis  requiring 
the  inversion  of  the  covariance  matrix  in  this  five  dimensional 
original  data  space,  they  would  not  be  surprised  to  find  that 
the  matrix  to  be  inverted  is  singular.  Similarly,  one  should 


not  expect  to  be  able  to  find  five  eigenvectors  associated 
with  this  data  set.  Table  1  shows  an  example  using  the 
Givens-Householder  technique  where  the  substitution  of  the 
covariance  matrix  associated  with  these  three  five  component 
data  vectors  into  conventional  eigenvector  routines  will 
lead  to  five  eigenvalues  and  five  associated  eigenvectors. 
The  two  smallest  eigenvalues  and  their  associated  eigen¬ 
vectors  must  be  meaningless  and  should  be  discarded. 


TABLE  1  -  SAMPLE  VECTORS  AMD  EIGENVECTORS  DERIVED  USING 
GIVENS-HOUSEHOLDER  TECHNIQUE 

3  -  INPUT  VECTORS 

VI  »  -10.  0.  0.  0.  0.2 

V2  s  0.  1.  0.  0.  0. 

V3  =  10.  *  -1.  0.  0.  -0.2 


5  -  EIGENVECTORS  AND  CORRESPONDING  EIGENVALUES: 

EIGENVECTORS  EIGENVALUE 


El 

— 

0.9985 

-0.0503 

-0.0200 

-4.3E-12 

0.0 

200. 

E2 

s 

0.0503 

-0.9987 

5.6E-8 

-2.2E-17 

-5.7E-17 

1.5 

E3 

=■ 

•2.8E-20 

-7.0E-17 

2.2E-10 

-1.0 

-1.0 

6.0E-18 

E4 

_ 

0. 

0. 

0. 

0. 

1. 0B-11 

2.8E-16 

-0.9998  -2.2E-10  1.4E-15 


E5  ■  -0.0200  -0.0010 


0.0 


If  we  now  introduce  two  additional  cases  which  are 
linearly  dependent  on  the  original  three  cases,  there  will  be 
no  charge  in  the  above  described  situation  except  that  in 
general  the  eigenvalues  and  the  eigenvectors  will  change. 

However,  if  these  two  linearly  dependent  eigenvectors  are 
noisy  they  may  introduce  additional  positive  eigenvalues, 
making  the  uses  believe  that  there  are  more  than  three 
meaningful  eigenvectors,  even  though  a  maximum  of  three 
eigenvectors  can  have  any  meaning.  We  would  hope  that  these 
eigenvectors  were  those  associated  with  the  largest  eigenvalues, 
however,  this  can  not  be  assured.  If  each  of  the  data  vectors 
were  similar  such  that  the  first  eigenvector  explained  almost 
all  of  the  variation  and  the  second  and  third  eigenvectors 
only  explained  a  small  amount  of  the  variation,  the  eigenvalues 
of  the  noise  generated  eigenvectors  may  exceed  the  eigenvalues 
of  the  true  eigenvectors.  Thus,  the  ill-conditioned  data  which 
leads  to  roost  problems  appearing  as  singular  matrices  in  data 
space  analysis  when  combined  with  noisy  data  will  lead  to 
the  generation  of  false  eigenvectors  when  one  attempts  to 
derive  eigenvectors  with  most  modern  iterative  techniques. 

Thus,  the  data  conditioning  necessary  to  insre  successful 
results  in  the  data  space  analysis  is  equally  important  to 
deriving  the  eigenvectors. 

When  dealing  with  real  data  especially  with  real  data 
defined  by  a  large  number  of  observations  especially  where  the 
number  of  cases  is  only  slightly  greater  than  the  number  of 
measurements  defining  each  of  the  observations,  linear  depend¬ 
ence  of  cases  within  this  data  and  noise  may  create  these 
problems  even  where  one  would  not  expect  them.  Noisy  data 
further  aggravates  the  problem  by  decreasing  the  number  of 
independent  observations.  Large  sets  of  real  data  where 
each  data  vector  is  itself  a  high  dimensional  vector  are 
particularly  susceptable  to  a  noise  induced  linear  dependence. 
That  is,  although  a  given  observation  may  in  principal  be 
independent  of  all  other  observations  it  may  be  sufficiently 
similar  that  the  difference  between  it  and  another  observation 
is  within  the  noise  or  the  inaccuracies  of  the  measurements. 

When  this  occurs,  it  can  dramatically  decrease  the  number  of 
independent  cases  available  and  it  is  often  difficult  to 
determine  this  effect  apriori  by  examination  of  the  data  or 
even  the  physics  of  the  process.  Since  this  noise  induced  linear 
dependence  will  also  reduce  the  total  number  of  eigenvectors 
which  can  be  expected  from  the  covariance  matrix,  it's  effect 
can  appear  in  exactly  the  same  way  as  the  simple  example  given 


above.  Thus,  we  see  that  in  dealing  with  real  data,  the  use 
of  poorly  conditioned  data  in  conventional  eigenvector 
derivation  procedures  can  lead  to  a  large  percentage  of  the 
eigenvectors  being  generated  from  measurement  inaccuracies 
or  other  noise  and  having  no  real  relationship  to  the  data. 
Furthermore,  this  has  been  accomplished  with  a  great  deal 
of  unnecessary  effort  on  the  part  of  the  computer.  This 
unnecessary  effort  has  increased  both  the  core  size  and 
running  time  required. 

ADAPT  EIGENVECTOR  TECHNIQUE 

The  ADAPT  technique  to  circumventing  the  conventional 
problems  in  deriving  eigenvector  representations  is  to 
precondition  the  data  matrix  by  a  proprietary  procedure  which 
eliminates  the  above  described  problems  and  is  mathematically 
equivalent  to  orthogonal! zing  the  matrix  without  optimizing. 
This  preconditioned  data  is  then  used  to  derive  the  Karhunen- 
Loeve  expansion  appropriate  to  the  original  data. 
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APPENDIX-  C 


ILLUSTRATION  OF  ADa£t  INDEPENDENT  EIGENSCREENING  TECHNIQUE 


Tables  1  through  3  present  typical  outputs  from  the 
ADAPT  independent  eigenscreening  programs.  These  tables 
illustrate  the  development  of  a  regression  algorithm  using 
independent  eigenscreening  for  estimating  the  change  in 
longitude  of  a  tropical  storm  24  hours  after  its  observation. 
Before  using  these  tables  to'  illustrate  the  independent 
eigenscreening  technique,  we  will  describe  the  information 
presented  on  the  tables.  The  tables  consist  of  ten  columns, 
each  of  these  columns  defines  one  parameter  of  interest. 

To  understand  the  information  presented,  we  must  recall 
that  the  procedure  used  is  to  divide  the  training  cases  into 
two  groups,  the  first  group  to  be  used  as  training  and  the 
second  group  as  independent  test.  For  example,  consider  a 
set  o.\  60  training  cases,  we  might  take  the  first  50  as  the 
training  and  the  last  10  as  independent  test.  The  algorithms 
would  then  be  derived  using  the  first  50  cases  and  tested 
against  the  iast  10.  When  this  is  completed,  a  different  set 
of  50  training  cases  and  10  independent  would  be  used.  For 
example,  we  might  now  take  Cases  1  through  40  and  51  through  60 
as  training  data  and  test  the  results  against  Cases  41  through 
50.  After  completion  of  the  second  set  of  algorithms  and 
independent  tests,  we  could  repeat  the  procedure  four  n.ore 
times.  This  would  yield  six  different  training  algorithms 
and  sets  of  10-  independent  tests  on  each  of  the  six  algor  ns 

for  a  total  of  60  independent  test  oases.  Thus,  beginning 
with  a  total  set  of  60  cases  this  procedure  would  result  in 
50  cases  for  training  and  60  independent  test  cases. 

The  selection  of  six  sets  of  algorithms  and  the  composition 
of  each  set  are  input  parameters  and  are  selected  based  on 
the  physics  of  the  problem.  The  penalty  is  that  we  need  to  develop 
six  sets  of  algorithms.  Using  conventional  techniques  the 
cost  of  this  procedure  would  be  prohibitive  for  most  real 
problems,  but  with  the  ADAPT  procedures  we  can  take  this 
approach.  As  a  result  of  this  approach,  we  have  a  performance 
for  the  independent  test  cases,  in  this  case,  the  correlation 
coefficient  given  in  the  third  column  of  the  tables  and  labeled 
RHOZVT.  We  also  have  a  learning  correlation  coefficient  for 
each  of  the  six  algorithms  developed.  The  average  of  these 


coefficients  is  given  in  the  fourth  column  of  the  table  and 
labeled  RHOVL.  We  may  also  compute  the  standard  deviation 
of  this  learning  correlation  coefficient  and  if  the  algorithm 
is  stable  we  would  expect  that  the  standard  deviation  of  the 
learning  correlation  coefficients  would  be  small  compared 
to  the  average  learning  correlation  coefficient.  Thus, 
we  define  the  ratio  of  the  standard  deviation  to  the  average 
correlation  coeffient  of  the  learning  data  as  the  learning 
stability.  This  is  provided  in  the  tenth  pr  last  column  of 
the  table  under  the  title,  "Learn  Stab*'. 

Since  we  have  developed  six  algorithms  and  each  algorithm 
has  a  number  of  terms  in  it  equal  to  the  dimensionality  of 
the  analysis  given  by  Column  2  in  the  tables,  we  can  also 
examine  the  stability  of  each  term  in  the  algorithm  in  the 
same  way  as  we  examine  the  stability  of  the  performance. 

This  stability  is  given  in  the  tenth  column  of  the  table  under 
the  title  of  MAXSIG/MEAN.  The  value' given  is  the  value  of 
the  worst  stability  of  any  term  in  the  algorithm,  the  number, 
NO,  is  given  for  some  outputs  and  is  the  term  in  the  algorithm 
which  has  this  worst  stability.  When  the  stability  exceeds 
an  input  threshold  parameter,  the  entire  stability  for  the 
algorithm  is  printed  out  (on  a  separate  page  from  this  summary 
table)  so  that  the  user  may  examine  it.  Our  experience  has 
shown  that  the  learning  stability  is  an  almost  certain  test 
of  having  obtained  the  ’-oVerdetermined  solution.  Experience 
with  a  number  of  different  types  of  data  and  problems  suggests 
that  the  stability  parameter,  MAXSIG/MEAN  must  be  less  than 
0.4  to  0.5  for  all  terms.  The  most  complete  test  of  this  is 
reported  in  Section  2.4,  performance  evaluation  methodology 
of  the  body  of  this  report. 

The  fifth  column  in  the  tables  labeled,  ACT-EST,  gives 
the  average  error  based  on  the  independent  testing.  The 
three  columns  labeled,  SDZV,  SIGRATL,  or  SIGRZVDT,  list  the 
standard  deviations  and  ratios  of  standard  deviations  which 
we  have  found  useful  in  assisting  in  the  understanding  of  the 
performance  of  the  algorithms  which  have  been  developed. 

The  first  column  of  Table  1  showing  the  potentially  useful 
eigendirections  provides  a  definition  of  which  eigendirections 
•are  being  used  in  any  algorithms  developed.  In  order  to 
provide  brevity  in  the  table,  only  the  last  eigendirection 
.added  is  listed.  Thus,  the  bottom  row  of  the  first  column 
of  this  table  has  a  value:  "2",  this  indicates  that  the 
first  eigendirection  which  was  useful,  was  the  second  eigen- 
direction  and  that  the  algorithms  developed  to  determine  this 


wore  developed  in  the  one  dimensional  space  (i.e.  Column  2 
headed  NR  has  a  value  of  NR  s  1)  consisting  of  the  second 
eigendi recti on.  The  second  row  up  shows  that  Column  1 
contains  the  value  "4".  This  indicates  that  the  fourth 
eigendirection  was  the  second  useful  eigendirection  and  the 
algorithm  to 

determine  this  was  derived  in  the  two  dimensional  eigenvector 
space  consisting  of  the  second  and  foi:rth  eigenvectors.  Thus, 
if  one  were  to  read  at  the  tenth  row  up  from  the  bottom  this 
is  an  algorithm  developed  in  a  ten  dimensional  space  consisting 
of  all  of  the  eigendirections  shown  in  the  first  column  from 
the  tenth  row  down  to  the  first  row. 

Now  that  we  have  looked  at  the  format  of  Tables  1 
through  3,  we  shall  discuss  their  meaning.  Table  1  summarizes 
all  of  the  eigendirections  which  have  bee^  selected  for 
retention  based  on  the  input  parameters  given  by  the  user. 

That  is,  the  user  is  allowed  to  input  a  criteria  for  both  the 
Independent  test  results  and  the  stability  which  must  be 
satisfied  in' order  to  retain  a  given  eigendirection  as  a 
result  of  this  screening.  Table  2  shows  the  same  results  for 
thc-';::c  eigendirections  which  have  been  rejected  based  upon 
the  criteria.  In  general,  it  is  our  practice  on  the  first 
screening  run  to  put  in  very  weak  constraints  on. the  retention 
of  eigendirections  so  that  we  retain  any  eigendirection  which 
has  any  possibility  of  being  useful  in  ..the  analysis  in  this 
first  pass.  For  this  study,  this  yielded  a  total  of  19  eigen¬ 
directions  which  appeared  to  have  some  usefulness  for  the 
task  at  hand.  We  then  repeat  the  screening  procedure  in 
reverse.  That  is,  we  start  with  all  of  the  potentially  useful 
eigendirections  and  sequentially  delete  one  of  the  eigen¬ 
directions  and  determine  whether  its  deletion  has  Improved 
or  decreased  the  performance  of  the  algorithm.  It  sometimes 
requires  several  steps  and  Table  3  presents  the  results  of  the 
last  step  of  this  analysis.  This  table  has  the  same  format 
as  Tables  1  and  2.  In  general,  the  criteria  utilized  are 
somewhat  more  stringent  in  these  final  steps.  Examination 
of  this  table  immediately  shows  the  most  successful  algorithm, 
is  the  ten  dimensional  algorithm  using  eigendirections  53,  42, 
34,  31,  20,  11,  8,  7,  6  and  4.  Note,  that  at  dimensionalities 
greater  than  or  equal  to  eleven  both  the  error  and  the  stability 


of  the  termsin  the  algorithm  (MAX  SIG/MEAN)  deteriorate. 

This  table  then  completes  the  screening  process. 

In  summary,  because  of  the  efficiency  of  the  ADAPT 
process,  we  have  been  allowed  to  make  our  decisions  as  to  the 
value  of  retaining  eigendirections  based  on  independent  tests 
as  well  as  on  the  stability  of  the  terms  in  the  algorithm  and 
the  performance  of  the  algorithm.  Furthermore,  we  have  not 
had  to  concern  ourselves  with  the  possibiU  ty  that  a  given 
direction  is  being  retained  because  of  linear  dependence  on 
another  eigendirection  because  of  the  orthogonal  properties 
of  the  eigendirections.  Although  this  example  has  been  given 
for  a  regression  analysis,  our  programs  are  completely 
operational  and  provide  exactly  the  same  results  using  similar 
outputs  for  a  Fisher  classifier.  Similar  procedures  can  be 
prepared  for  any  linear  classifier. 
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APPENDIX  D  -  CHARACTERISTICS  OF  ENGINE  HEALTH 
EIGENVECTORS 


The  purpose  of  this  appendix  is  to  document  the 
significant  features  of  the  eigenvector  expansion  which 
were  not  documented  in  the  main  body  of  the  report.  Two 
other  appendices,  A  and  B,  provide  the  definition  of  the 
significance  of  the  plots  included  in  this  appendix.  The 
average  vector  used  to  reduce  the  data  to  zero  mean,  the 
information  energy  plot  and  the  projection  on  the  first  two 
eigendirections  were  presented  in  the  body  of  the  report 
as  Figures  11,  12  and  13,  respectively  of  the  body  of  this  report. 

Figures  Dl,  D2  and  D3  continue  the  projections  on  the 
next  six  eigendirections.  They  are  plotted  in  the  same  format 
and  using  the  same  symbols  as  Figure  13. 

Figures  D4-D7  present  plots  of  the  data  space  represent¬ 
ation  of  typical  eigenvectors.  Figures  D4  show  the  dominant 
and  intermediate  eigenvectors.  The  dominant  eigenvector  is 
the  first  eigenvector.  The  largest  contributes  to  this  eigen¬ 
vector  is  the  engine  test  cell  data.  Eigenvectors  2-27  may 
be  considered  intermediate  order  eigenvector  and  have  the 
characteristic  that  NQAP  and  test  cell  data  are  of  similar 
importance .  The  28th  eigenvector  and  those  shown  in  Figure  D-6 
may  be  called  higher  order  eigenvectors  and  are  dominated  by 
the  NOAP  data  with  very  little  contribution  from  the  test  cell. 

Referring  to  Figure  12  and  the  analysis  given  by  Overland* 
suggests  that  possibly  only  the  first  eigendirection  is  physically 
unique.  However,  even  the  non-unique  eigendirections  are  suitable 
as  features  in  studies  such  as  this.  Furthermore,  since  any  new 
intermediate  or  higher  order  eigendirection  must  be  constructable 
by  using  a  linear  conibination  of  the  "near"  eigenvectors,  the 
general  features  of  the  intermediate  and  higher  order  eigenvector 
sets  also  remain  unique. 


*  Overland,  James  E.;  “A  Significance  Test",  Mon  Wea  Review, 
Vol  110,  No-1. 
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FIG-D2  PROJECTION  ON  FIFTH  AND 
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FIGURE  D4  -  DEFINITION  OF  DOMINANT  EIGENVECTOR  OR  EIGENDIRECTION 
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FIGURE  D4  (CONT'D)  -  DEFINITION  OF  DOMINANT  EIGENVECTOR  OR  EIGENDIRECTION 
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EIGENVECTOR  NUMBER-©  -  FOR 
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FIGURE  D6  -  TYPICAL  HIGHER  ORDER  EIGENVECTORS 
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